Deleting old volumes

[Valentin Dzhorov]

I understand the core philosophy of bareos and the fact that as it predecessor bacula are meant to backup tapes. My storage however is consisted of HDDs only and there is no need of changing tape drives. I am reading through the documentation and I thought I got it right, but apparently I didn't. My goal is the following: I want to have full 7 days of backups at any given time for each client that is backed up. A job is configured like so:

[root@directorbareos ~]# cat /etc/bareos/bareos-dir.d/job/ufo1.delta.bg.conf 

Job {

  Name = example.com

  Type = Backup

  Level = Full

  Client = example.com

  FileSet= LinuxAll

  Messages = Standard

  Storage = examplestorage

  Pool = LinuxAll

  Schedule = Weekly_10pm

  Priority = 20

  Allow Mixed Priority = yes

  Reschedule On Error = yes

  Reschedule Interval = 180

  Reschedule Times = 2

  Run Script {

    Console = ".bvfs_update jobid=%i"

    RunsWhen = After

    RunsOnClient = No

  }

}

And my volume is configured like so:

Pool {

  Name = example.com

  Pool Type = Backup

  Recycle = yes

  Auto Prune = yes

  Volume Retention = 7 days

  Label Format = "example.com-"

  Volume Use Duration = 14d

  Recycle Oldest Volume = yes

  Storage = examplestorage

}

So, how can I achieve having full 7 days of backup the most efficient way without wasting space and having the old files automatically deleted from the volume, not just the catalog? 

[Brock Palen]

What maters here more is your retention time on the client/jobs so you make sure you can do the restore. So check all of those.

In your Pool config you volume use is longer than your retention. You probably want to swap those. So the volume stops being used for new backups before the volume is pulled out of your catalog.

Now your volume may not actually get recycled, in my experience Bareos tries to not throw out data unless it has to. So even if purged from catalog it will keep labeling new volumes. This can cause you to run out of space on disk volumes.

To address this, I have a job that runs daily that calls a script that forces updating the prunes on the volume and then truncates them (frees up disk space) which will force them to be reusable for the next needed volume.

Update for your pool name.

#!/bin/bash
for x in `echo "list volumes pool=AI-Consolidated" | bconsole | grep -v "list volumes" | grep AI-Consolid | awk -F\| '{print $3}'`
do
echo "prune volume=$x yes"
done | bconsole

# actaully free up disk space
echo "truncate volstatus=Purged pool=AI-Consolidated yes" \
| bconsole


So check those over retention times, and then if your volumes are getting piled up with old data try the second part.

Brock Palen

> --
> You received this message because you are subscribed to the Google Groups "bareos-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/bfdceca4-6811-4ef2-9f55-80a598b286ac%40googlegroups.com.

[Valentin Dzhorov]

Thank you for the response Brock! I think I understand the basic concept more - so if I want 7 days of backups, I should ensure that my records in the catalog stay longer than that, hence I should increase my volume retention, job retention and file retention? I've now incorporated the following config for each of my pools:

Pool {

  Name = example.com

  Pool Type = Backup

  Recycle = yes

  Auto Prune = yes

  Volume Retention = 14d

  File Retention = 14d

  Job Retention = 14d

  Label Format = "example.com-"

  Volume Use Duration = 7d

  Recycle Oldest Volume = yes

  Action On Purge = truncate

  Storage = examplestorage

  Maximum Volumes = 2

}

The idea here is that I can have 14 days of records for jobs, volumes and files in the catalog. I have set the use duration of each volume to 7 days and maximumum of 2 volumes so that I can have automatic rotation of volumes. Do you think I am on the right track with this config?

For the second part you've suggested - I understand what the truncate on volume does, however I do not understand how to use it automatically. Say I keep 2 volumes per client as per the config I've described above. How can I automatically truncate the oldest volume at the end of these 14 days retention period for jobs, files and volumes? The script you've provided does not know the age of the volume unfortunately.

[Spadajspadaj]

Firstly, you cannot have bareos delete files without dirty tricks. It can truncate volumes on purge as someone already pointed out.

If I were you and wanted to have fixed number of backups regardless of any other parameters, I'd go for maximum volume jobs=1 and apropriate retention and Maximum Volumes. Then I'd go for a separate pool for each client.

This way you'd have a fixed number of volumes in rotation, you'd have a separate media file for each job and with appropriate retention settings you'd recycle oldest volume each time.

One caveat - if you happen to have a job return an error state and want to rerun a job earlier than it's normally scheduled, you'd have more volumes used than you planned so you'd have to manually purge the volume containing errored job.