Always Incremental example?

darkadept

unread,

Nov 17, 2016, 11:59:16 AM11/17/16

to bareos-users

I'm new to bareos and I'm trying to implement the "Always Incremental" backup strategy.

I have about 250gb of data I'm backing up onto a NAS (mounted via CIFS).

I can't find any other documentation or examples about Always Incremental except for what it mentions in the docs. I think I have configured it as to what it says but I'm not sure. Does this look correct?

My assumption is that this configuration will:
* Do a full backup initially
* Do incremental backups every weekday at 21:00
* Consolidate (every weekday) the incrementals older than 7 days
* Always keep at least 7 incrementals
* Consolidate job only consolidates with the Full backup every 14 days, otherwise it consolidates just the incrementals (not sure if this is the correct assumption of how "Always Incremental Max Full Age" works.)
* Only consolidates 1 per consolidate job (if multiple clients).
* Creates a virtual full backup on the 1st saturday of every month.

-------------------------------------------------
On my Storage Daemon I have two devices configured:

Device {
Name = AI-Consolidated
Media Type = File
Archive Device = /mnt/nas/bareos/consolidated
LabelMedia = yes;
Random Access = yes;
AutomaticMount = yes;
RemovableMedia = no;
AlwaysOpen = no;
Description = "Used for AI consolidated backups"
}

Device {
Name = AI-File
Media Type = File
Archive Device = /mnt/nas/bareos/file
LabelMedia = yes;
Random Access = yes;
AutomaticMount = yes;
RemovableMedia = no;
AlwaysOpen = no;
Description = "Used for AI regular"
}

-------------------------------------------------
On my Director I have the following three pools:

Pool {
Name = AI-Incremental
Pool Type = Backup
Recycle = yes
AutoPrune = no
Maximum Volume Bytes = 50G
Label Format = "AI-Incremental-"
Volume Use Duration = 23h
Storage = MyStorage-AI-File
}

Pool {
Name = AI-Consolidated
Pool Type = Backup
Recycle = yes
AutoPrune = no
Maximum Volume Bytes = 50G
Label Format = "AI-Consolidated-"
Volume Use Duration = 23h
Storage = Steinbach-AI-Consolidated
Next Pool = AI-Longterm
}

Pool {
Name = AI-Longterm
Pool Type = Backup
Recycle = yes
AutoPrune = yes
Volume Retention = 10 years
Maximum Volume Bytes = 50G
Label Format = "AI-Longterm-"
Volume Use Duration = 23h
Storage = MyStorage-AI-File
}

-------------------------------------------------
I also have the following two Schedules defined:

Schedule {
Name = AI-Cycle
Run = Incremental mon-fri at 21:00
}

Schedule {
Name = AI-Longterm
Run = Full 1st sat at 21:00
}

-------------------------------------------------
And finally I have the following three Jobs defined:

Job {
Name = Backup-data
Client = myclient
FileSet = MyFileSetData

Always Incremental = yes
Always Incremental Job Retention = 7 days # older than this will be merged into new Virtual backup
Always Incremental Keep Number = 7 # will always keep this many incrementals
Always Incremental Max Full Age = 14 days
Accurate = yes
Pool = AI-Incremental
Full Backup Pool = AI-Consolidated

Type = Backup
Level = Incremental
Schedule = AI-Cycle
Messages = Standard
Priority = 10
Write Bootstrap = "/var/lib/bareos/%c.bsr"
}

Job {
Name = Consolidate-data
Type = Consolidate
Accurate = yes
JobDefs = DefaultJob
Max Full Consolidations = 1
}

Job {
Name = VirtualLongtermFull
Client = bareos-fd
FileSet = SelfTest
Schedule = AI-Longterm
Type = Backup
Level = VirtualFull
Pool = AI-Consolidated
Messages = Standard
Priority = 13
Run Script {
console = "update jobid=%i jobtype=A"
Runs When = After
Runs On Client = No
Runs On Failure = No
}
}

Seitan

unread,

Dec 30, 2016, 2:26:03 AM12/30/16

to bareos-users

Yes, the documentation is not informative on how to configure "Always incremental" backups. Using information from documentation leads to non-working consolidation jobs.
Full example would be really appreciated.

Hosted Power

unread,

Jan 3, 2017, 4:53:23 AM1/3/17

to bareos-users

Op vrijdag 30 december 2016 08:26:03 UTC+1 schreef Seitan:

> Yes, the documentation is not informative on how to configure "Always incremental" backups. Using information from documentation leads to non-working consolidation jobs.
> Full example would be really appreciated.

Indeed, having lots of issues with it too :|

Михаил Калинин

unread,

Jan 31, 2018, 7:13:28 AM1/31/18

to bareos-users

How prune consolidated jobs?

Dan

unread,

Jan 31, 2018, 3:50:06 PM1/31/18

to bareos-users

On Thursday, November 17, 2016 at 11:59:16 AM UTC-5, darkadept wrote:
> I'm new to bareos and I'm trying to implement the "Always Incremental" backup strategy.
>

> My assumption is that this configuration will:

> * Do a full backup initially
> * Do incremental backups every weekday at 21:00
> * Consolidate (every weekday) the incrementals older than 7 days
> * Always keep at least 7 incrementals
> * Consolidate job only consolidates with the Full backup every 14 days, otherwise it consolidates just the incrementals (not sure if this is the correct assumption of how "Always Incremental Max Full Age" works.)
> * Only consolidates 1 per consolidate job (if multiple clients).
> * Creates a virtual full backup on the 1st saturday of every month.

----------------------------------
You are mostly correct in your configuration. The documentation almost gets you to a working AI implementation, but there are a few gotchas that have to be worked through. It's hard keep my answer clear by marking up each of your conf files, so I'll post you a configuration that I've had in production for a while as complete working example. I'll use the notations BOF and EOF for beginning of file and end of file, respectively. Lines containing those notations should not be included in you conf files.

This is a LONG POST. Bear with me.

The below configuration will ...
* Do incremental backups 20:00 every day. By definition the 1st will be a full backup.
* Consolidate at 21:00 every 4th day starting on the 3rd day of the month.
* Only consolidate incrementals older than 7 days
* Keep at least 7 incrementals
* Consolidate with the Full backup every 14 days. (I don't consolidate very day, so my full gets picked up on the first consolidate job on or after 14 days.

* Only consolidates 1 per consolidate job (if multiple clients).

* Creates a virtual full backup at noon on the 1st day of the month.

-------------------------------------------------
On my Storage Daemon I have the following:

STORAGE:
BOF - storage/bareos-sd.conf
Storage {
Name = bareos-sd
Maximum Concurrent Jobs = 5
Device Reserve By Media Type = yes # This entry is REQUIRED for successful consolidation. See FileStorageCons.conf.
}
EOF

DEVICES:
BOF - device/FileStorage.conf
Device {
Name = FileStorage
Media Type = File
Archive Device = /bareosFiles/BareosBackups/storage

LabelMedia = yes
Random Access = yes
AutomaticMount = yes
RemovableMedia = no
AlwaysOpen = no

Description = "File device of for general use."
Maximum Concurrent Jobs = 5
}
EOF

BOF - device/FileStorageCons.conf
Device {
Name = FileStorageCons1 # Multiple devices, different name and same media type, are configured to support simultaneous use.
Media Type = FileCons
Archive Device = /bareosFiles/BareosBackups/storage