Virtual Full Backups in Always Incremental scheme disk space requirements

[James Harrison]

Hello,

I have been able to get Always Incremental backups using Max Full Age working per the documentation

https://docs.bareos.org/TasksAndConcepts/AlwaysIncrementalBackupScheme.html

as well as this example:

https://groups.google.com/forum/?hl=en#!searchin/bareos-users/Always$20incremental$20example%7Csort:date/bareos-users/i5H0ScDWA4Y/JUBDGex-AQAJ

My question is on when the Max Full Age expires and Bareos tries to do a consolidation/Full Virtual Backup, it needs to keep around the old full backup (to read from it) as well as write to the new full backup, which means I need to effectively have twice the space on my archive drive than I do for my data drive; i.e. if I have 6TB, I would need a 12TB drive for backup. Is this a correct assumption? If so, is there a way to do full virtual backups more efficiently?

I have seen some other postings about jobs/scripts that cleanup the unused volumes, which are cool, but one would still need 2x the space so that the full virtual backup operation can successfully complete.

I have tried setting my Maximum Volume Bytes to a number where I would get a number of different volumes, along with playing with options like Action On Purge = Truncate and Purge Oldest Volume = yes but to no avail.

Is there anyway to configure Bareos to purge/remove/delete a volume as soon as it's read/copied? From my experiments, it looks as though Bareos needs to read all the backup volumes first before moving the data over during a Full Virtual backup

[James Harrison]

To detail my experiment that I've mentioned above, I have a test case with about 200MB of data. I then created a chron job that will copy small amounts of data every 5 minutes. Incremental backups occur every 5 minutes as well, so when I look to restore my files, I can map exactly which files line up with which incremental jobs to make sure that the scheme is working properly. Consolidation happens every half hour, full backups every hour

Job {
    Name = "BackupClient1"
    Client = bareos-fd
    FileSet="FileSetTest"
    
    Type = Backup
    #Level = Incremental
    #Storage = File
    Messages = Standard
    
    #JobDefs = "DefaultJob"

    # Always incremental settings
    AlwaysIncremental = yes
    AlwaysIncrementalJobRetention = 27 min  #incrementals older than this will be consolidated
    AlwaysIncrementalMaxFullAge = 57 min    #if the full backup is older than this the consolidations will be rolled into it

    Accurate = yes

Since the data I'm backing up + consolidations comes to about ~240MB, I gave the AI-Consolidated pool a space of about 360MB

  Maximum Volume Bytes = 15 MB  

  Maximum Volumes = 24

When Bareos tries to do a second full backup, it will start to create more AI-Consolidated volumes, but it eventually runs out of space per the Max Volume number limit. The backup job gives me the message

bareos-sd JobId 35: Job BackupClient1.2020-10-09_19.10.26_46 is waiting. Cannot find any appendable volumes.
Please use the "label" command to create a new Volume for:
Storage: "FileStorageCons1" (C:/bareos-storage)
Pool: AI-Consolidated
Media type: FileCons

And then hangs forever.

Ideally I'd like to recycle/prune/purge the volumes as they are read to conserve space. If anyone has any ideas on how to do this, please let me know.