Doing base backups without wals with barman

[Jaanus Rauk]

Hi,,

I have a very active database and I don't need to back up it latest state.

Is there any way to backup it without streaming wals?

(pg_basebackup --wal-method=none)

Thanks,

Jaanus

[Luca Ferrari]

On Sun, Mar 14, 2021 at 10:54 AM Jaanus Rauk <jaanu...@gmail.com> wrote:
>
> (pg_basebackup --wal-method=none)

Isn't that the solution?
As far as I know barman requires archiving, therefore it should refuse
to do backup while not archiving.
I would try to do a backup and remove archiving immediatly after, but
I would not recommend this.
Luca

[Jaanus Rauk]

Yes i can use pg_basebackup.
But then i need to create unique tooling-scripts  for  just one database.

[Luca Ferrari]

On Sun, Mar 14, 2021 at 12:28 PM Jaanus Rauk <jaanu...@gmail.com> wrote:
>
>
> Yes i can use pg_basebackup.
> But then i need to create unique tooling-scripts for just one database.

Yes, I undertsand, but I'm pretty sure barman will refuse to backup a
cluster without having archiving set. Or at least, I've not tried to
issue a backup until `check` is satisfied.
Do you think it is possible to backup and then stop the archiving
(i.e., set archive_command to a dummy value)?

Luca

[Jaanus Rauk]

The streaming and basebackup causes so much disk IO, so even when basebackup succeedes, the barman gets some timeout and marks the backup as failed:

Finalising the backup.
ERROR: Backup failed copying files.
DETAILS: Could not find backup_id 20210314T015848

but yeah , i got the info - it seems some custom scripting is easier.

Thanks!

[Luca Ferrari]

On Sun, Mar 14, 2021 at 4:50 PM Jaanus Rauk <jaanu...@gmail.com> wrote:
>
>
> The streaming and basebackup causes so much disk IO, so even when basebackup succeedes, the barman gets some timeout and marks the backup as failed:
> Finalising the backup.
> ERROR: Backup failed copying files.
> DETAILS: Could not find backup_id 20210314T015848
>
> but yeah , i got the info - it seems some custom scripting is easier.

I'm just running a backup thru barman configured for streaming but not
for archiving:

% sudo -u postgres postgres -C archive_mode -D $PGDATA
on
% sudo -u postgres postgres -C archive_command -D $PGDATA
/usr/bin/true

$ barman backup miguel
Backup start at LSN: 0/C7000480 (0000000100000000000000C7, 00000480)
Starting backup copy via pg_basebackup for 20210315T081128
Copy done (time: 2 minutes, 13 seconds)
Finalising the backup.
Backup size: 508.9 MiB
Backup end at LSN: 0/CA000060 (0000000100000000000000CA, 00000060)

This has been done with streaming replication, but not any archiving.
If I stop the wal receiver barman refuses to do the backup.
So my idea could be to stop archiving, let streaming replication to
work, do the backup and kill the wal receiver once the backup has been
done. This should achieve something pretty close to the behavior your
are looking for, but I'm not sure it is going to fix your specific
problem.

Any chance you can limit the I/O during the backup to fix the problem
at its root?

Luca

[Niels Stevens]

Hey Luca,

I find myself in a similar situation where I'm only interested in performing pg_basebackups and not setting up wall receivers. It seems you are struggling with this problem too, and I wonder if you find a solution for this using Barman. Or ultimately choose another approach.

Thanks, Niels

[Luca Ferrari]

On Thu, Jun 10, 2021 at 8:28 PM 'Niels Stevens' via Barman, Backup and
Recovery Manager for PostgreSQL <pgba...@googlegroups.com> wrote:
> I find myself in a similar situation where I'm only interested in performing pg_basebackups and not setting up wall receivers. It seems you are struggling with this problem too, and I wonder if you find a solution for this using Barman. Or ultimately choose another approach.

Well, I was not interested in this kind of backup, and I stand correct
on my original answer: pg_basebackup --wal_method=none is not the
solution. In fact, that means your backup copy will be "incomplete"
(inconsistent is the right word), but if that is what you want...
My guess is you need to leave streaming replication activated during
the backup, and shut it down once the backup has completed
(inspecting the backup files via barman is a good way to see what you
have and what you don't need).
I don't see the point in making a backup without WALs, and if that is
the need, pg_basebackup is the way.

Maybe, barman developers could elaborate more on this.

Luca

[Abhijit Menon-Sen]

On Fri, Jun 11, 2021 at 11:46 AM Luca Ferrari <fluc...@gmail.com> wrote:
>
> Well, I was not interested in this kind of backup, and I stand correct
> on my original answer: pg_basebackup --wal_method=none is not the
> solution. In fact, that means your backup copy will be "incomplete"
> (inconsistent is the right word), but if that is what you want...
> My guess is you need to leave streaming replication activated during
> the backup, and shut it down once the backup has completed
> (inspecting the backup files via barman is a good way to see what you
> have and what you don't need).
> I don't see the point in making a backup without WALs, and if that is
> the need, pg_basebackup is the way.
>
> Maybe, barman developers could elaborate more on this.

I agree with this. A base backup without at least the WAL generated
during the backup is not complete/self-contained, and it is not
something Barman does or will support.

I do not recommend doing this at all, but if you insist on doing it,
you'll have to use `pg_basebackup --wal-method=none`. Please note that
without access to this WAL, postgres will refuse to start against the
backed-up directory. You'll get a FATAL error like "could not locate
required checkpoint record". Even the pg_basebackup manual page says
"Unless the method none is specified, it is possible to start a
postmaster in the target directory without the need to consult the log
archive, thus making the output a completely standalone backup."

-- Abhijit