how often and how should "barman backup" be run?

[Chris Withers]

Hi All,

I see that incoming WAL files are processed by "barman cron", but how often should "barman backup" be run?

How do people normally schedule these runs? cron again?

cheers,

Chris

[Cédric Villemain]

Hi Chris

> I see that incoming WAL files are processed by "barman cron", but how
> often should "barman backup" be run?
> How do people normally schedule these runs? cron again?

By cron, or sometime with a more elaborated backup manager (one use for
a full range of service for example). It can be a backupninja executing
'barman cron' or an EMC networker script (look at the archive for a
recent post on this subject, with a complete example). Feel free to
share what is convenient for your use case.

Cheers,
Cédric

PS: please keep me in CC

--
Cédric Villemain +33 (0)6 20 30 22 52
http://2ndQuadrant.fr/
PostgreSQL: Support 24x7 - Développement, Expertise et Formation

[Chris Withers]

Hi Cédric,

On 18/06/2015 16:44, Cédric Villemain wrote:
>
> I see that incoming WAL files are processed by "barman cron", but how
> often should "barman backup" be run?
> How do people normally schedule these runs? cron again?
> By cron,

I think I just needed to read more of the docs, am I right in thinking
that the once-a-minute run of 'barman -q cron' that the rpm
installations of barman set up will do all that's necessary to get full
backups created as per the retention policy?

Chris

[Cédric Villemain]

AFAIR, no.
Someone else to confirm that please ?

[Chris Withers]

On 18/06/2015 17:30, Cédric Villemain wrote:
> Le 18/06/2015 18:28, Chris Withers a écrit :
>
>> I think I just needed to read more of the docs, am I right in thinking
>> that the once-a-minute run of 'barman -q cron' that the rpm
>> installations of barman set up will do all that's necessary to get full
>> backups created as per the retention policy?
> AFAIR, no.
> Someone else to confirm that please ?
>

Oh?

The "Maintenance Mode" section on
http://docs.pgbarman.org/index.html#available_commands suggests that the
retention policy regarding full backups and the like will be enforced by
the 'barman cron' runs.

Are you saying they won't?

Chris

[Cédric Villemain]

I believe it is related to the deletion of the backups.
See "How they work" about retention policies.

[Gianni Ciolli]

The purpose of retention policies is not to make backups; it is to
delete them automatically when they become "deletable", to avoid
filling up the disk.

Backups are not performed automatically. If you want to implement a
similar functionality, you can easily add a separate cronjob line. In
this case, the last_backup_maximum_age setting can help in detecting
failures.

Best wishes,
Dr. Gianni Ciolli - 2ndQuadrant Italia
PostgreSQL Training, Services and Support
gianni...@2ndquadrant.it | www.2ndquadrant.it

[Chris Withers]

On 18/06/2015 18:16, Gianni Ciolli wrote:
>
> The "Maintenance Mode" section on
> http://docs.pgbarman.org/index.html#available_commands suggests that the
> retention policy regarding full backups and the like will be enforced by the
> 'barman cron' runs.
>
> Are you saying they won't?
> The purpose of retention policies is not to make backups; it is to
> delete them automatically when they become "deletable", to avoid
> filling up the disk.
>
> Backups are not performed automatically. If you want to implement a
> similar functionality, you can easily add a separate cronjob line. In
> this case, the last_backup_maximum_age setting can help in detecting
> failures.

This seems a little silly. If barman is already running a once-a-minute
cron, why doesn't that also take backups as necessary to meet the policy
specified in the config file?

Chris

[David Hancock]

To ME, it's not silly at all. Depending on the size and activity level, a

backup can be a time- and resource-consuming operation, and it makes sense

that you wouldn't want it happening at any arbitrary time that barman cron

determines it's needed.

[Gabriele Bartolini]

Hi Chris,

2015-06-18 19:46 GMT+02:00 Chris Withers <ch...@simplistix.co.uk>:

This seems a little silly.

That might definitely be the case. That is not perfect but, as for all the things you see in Barman, we try to work on priorities. In our initial prototype, we decided to rely on the best scheduler widely available with every Linux system: cron.

 

If barman is already running a once-a-minute cron, why doesn't that also take backups as necessary to meet the policy specified in the config file?

That is such a reasonable question to me. We have ideas of having cron managing the scheduling of Barman backups, but only if this generate added value to Barman. At the moment, system's cron gives enough flexibility to everyone using Barman without adding unnecessary complexity to Barman.

Also, in terms of devops capabilities, it is very easy to add a cron entry using a configuration manager and its scope is perfectly visible to system administrators. In a few words, given that this operation could be heavy on a system, having it managed at system administration level would still make sense for most of scenarios.

So, where the added value could be? It could be in managing different backup schemes, such as the GrandFather, Father, Son scheme - as I believe I have pointed out already in this list. In this context, Barman could schedule daily, weekly and monthly backups independently. How to configure it, how to monitor it, how to optimise it? These are questions that still need to be faced. But only if there is demand for these feature, we will add them to Barman.

Any unnecessary feature is - potentially - another possible source of bugs and we certainly do not want that.

Thank you for your brilliant questions!

Ciao,

Gabriele

--
 Gabriele Bartolini - 2ndQuadrant Italia - Managing Director

 PostgreSQL Training, Services and Support

 gabriele....@2ndQuadrant.it | www.2ndQuadrant.it