Newbie first steps making backups

[Tan Mientras]

Hi all

After reading https://docs.pgbarman.org/release/3.5.0/ and with some help of Mike Wallace, I'm supposed to have a barman instance setup and ready for backup. 

However, it seems it's not working as expected, or at least not as documentation describes. Here are some details.

/etc/barman.conf (no more lines uncommented)

[barman]
barman_user = barman
configuration_files_directory = /etc/barman.d
barman_home = /var/lib/barman
log_file = /var/log/barman/barman.log
log_level = DEBUG

/etc/barman.d/pg.conf (wysiwyg)
[pg2023-05-31]
description = "Our main PostgreSQL server"
conninfo = host=192.168.121.28 user=barman dbname=postgres
backup_method = postgres
streaming_archiver = on
streaming_conninfo = host=192.168.121.28 user=streaming_barman dbname=postgres
slot_name=barman
create_slot = auto
archiver = off

barman check pg2023-05-31 (everything is ok!)
Server pg2023-05-31:
PostgreSQL: OK
superuser or standard user with backup privileges: OK
PostgreSQL streaming: OK
wal_level: OK
replication slot: OK
directories: OK
retention policy settings: OK
backup maximum age: OK (no last_backup_maximum_age provided)
backup minimum size: OK (0 B)
wal maximum age: OK (no last_wal_maximum_age provided)
wal size: OK (0 B)
compression settings: OK
failed backups: OK (there are 0 failed backups)
minimum redundancy requirements: OK (have 0 backups, expected at least 0)
pg_basebackup: OK
pg_basebackup compatible: OK
pg_basebackup supports tablespaces mapping: OK
systemid coherence: OK
pg_receivexlog: OK
pg_receivexlog compatible: OK
receive-wal running: OK
archiver errors: OK

barman backup pg2023-05-31
Starting backup using postgres method for server pg2023-05-31 in /var/lib/barman/pg2023-05-31/base/20230607T091012
Backup start at LSN: 0/25000060 (000000010000000000000025, 00000060)
Starting backup copy via pg_basebackup for 20230607T091012

HANGS HERE

/var/log/barman/barman.log

2023-06-07 09:10:12,706 [289894] barman.config DEBUG: Including configuration file: pg.conf
2023-06-07 09:10:12,708 [289894] barman.cli DEBUG: Initialised Barman version 2.18 (config: /etc/barman.conf, args: {'color': 'auto', 'quiet': False, 'debug': False, 'format': 'console', 'command': 'backup', 'server_name': ['pg2023-05-31'], 'reuse_backup': None, 'retry_times': None, 'retry_sleep': None, 'jobs': None, 'wait': False, 'wait_timeout': None, 'func': <function backup at 0x7f2683153370>})
2023-06-07 09:10:12,734 [289894] barman.backup_executor DEBUG: The default backup strategy for postgres backup_method is: concurrent_backup
2023-06-07 09:10:12,734 [289894] barman.server DEBUG: Starting check: 'WAL archive'
2023-06-07 09:10:12,735 [289894] barman.server DEBUG: Starting check: 'empty incoming directory'
2023-06-07 09:10:12,735 [289894] barman.server DEBUG: Starting check: 'empty streaming directory'
2023-06-07 09:10:12,735 [289894] barman.server DEBUG: Starting check: 'PostgreSQL'
2023-06-07 09:10:12,835 [289894] barman.command_wrappers DEBUG: Command: ['/usr/bin/pg_receivewal', '--version']
2023-06-07 09:10:12,896 [289894] barman.command_wrappers DEBUG: Command return code: 0
2023-06-07 09:10:12,897 [289894] barman.command_wrappers DEBUG: Command stdout: pg_receivewal (PostgreSQL) 14.8 (Ubuntu 14.8-0ubuntu0.22.04.1)
2023-06-07 09:10:12,897 [289894] barman.command_wrappers DEBUG: Command stderr:
2023-06-07 09:10:12,901 [289894] barman.wal_archiver DEBUG: Look for 'barman_receive_wal' in 'synchronous_standby_names': ['']
2023-06-07 09:10:12,901 [289894] barman.wal_archiver DEBUG: Synchronous WAL streaming for barman_receive_wal: False
2023-06-07 09:10:12,902 [289894] barman.command_wrappers DEBUG: Command: ['/usr/bin/pg_basebackup', '--version']
2023-06-07 09:10:12,958 [289894] barman.command_wrappers DEBUG: Command return code: 0
2023-06-07 09:10:12,959 [289894] barman.command_wrappers DEBUG: Command stdout: pg_basebackup (PostgreSQL) 14.8 (Ubuntu 14.8-0ubuntu0.22.04.1)
2023-06-07 09:10:12,959 [289894] barman.command_wrappers DEBUG: Command stderr:
2023-06-07 09:10:12,961 [289894] barman.server DEBUG: Check 'PostgreSQL' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,962 [289894] barman.server DEBUG: Starting check: 'superuser or standard user with backup privileges'
2023-06-07 09:10:12,962 [289894] barman.server DEBUG: Check 'superuser or standard user with backup privileges' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,962 [289894] barman.server DEBUG: Starting check: 'PostgreSQL streaming'
2023-06-07 09:10:12,962 [289894] barman.server DEBUG: Check 'PostgreSQL streaming' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,962 [289894] barman.server DEBUG: Starting check: 'wal_level'
2023-06-07 09:10:12,963 [289894] barman.server DEBUG: Check 'wal_level' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,963 [289894] barman.server DEBUG: Starting check: 'replication slot'
2023-06-07 09:10:12,963 [289894] barman.server DEBUG: Check 'replication slot' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,963 [289894] barman.server DEBUG: Starting check: 'directories'
2023-06-07 09:10:12,963 [289894] barman.server DEBUG: Check 'directories' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,964 [289894] barman.server DEBUG: Starting check: 'retention policy settings'
2023-06-07 09:10:12,964 [289894] barman.server DEBUG: Check 'retention policy settings' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,964 [289894] barman.server DEBUG: Starting check: 'backup maximum age'
2023-06-07 09:10:12,964 [289894] barman.server DEBUG: Check 'backup maximum age' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,964 [289894] barman.server DEBUG: Starting check: 'backup minimum size'
2023-06-07 09:10:12,965 [289894] barman.server DEBUG: Check 'backup minimum size' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,965 [289894] barman.server DEBUG: Starting check: 'wal maximum age'
2023-06-07 09:10:12,965 [289894] barman.server DEBUG: Check 'wal maximum age' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,965 [289894] barman.server DEBUG: Starting check: 'wal size'
2023-06-07 09:10:12,965 [289894] barman.server DEBUG: Check 'wal size' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,966 [289894] barman.server DEBUG: Starting check: 'compression settings'
2023-06-07 09:10:12,966 [289894] barman.server DEBUG: Check 'compression settings' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,966 [289894] barman.server DEBUG: Starting check: 'failed backups'
2023-06-07 09:10:12,966 [289894] barman.server DEBUG: Check 'failed backups' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,966 [289894] barman.server DEBUG: Starting check: 'minimum redundancy requirements'
2023-06-07 09:10:12,967 [289894] barman.server DEBUG: Check 'minimum redundancy requirements' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,967 [289894] barman.server DEBUG: Starting check: 'pg_basebackup'
2023-06-07 09:10:12,967 [289894] barman.server DEBUG: Check 'pg_basebackup' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,967 [289894] barman.server DEBUG: Starting check: 'pg_basebackup compatible'
2023-06-07 09:10:12,968 [289894] barman.server DEBUG: Check 'pg_basebackup compatible' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,969 [289894] barman.server DEBUG: Starting check: 'pg_basebackup supports tablespaces mapping'
2023-06-07 09:10:12,972 [289894] barman.server DEBUG: Check 'pg_basebackup supports tablespaces mapping' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,973 [289894] barman.server DEBUG: Starting check: 'configuration'
2023-06-07 09:10:12,973 [289894] barman.server DEBUG: Starting check: 'systemid coherence'
2023-06-07 09:10:12,973 [289894] barman.server DEBUG: Check 'systemid coherence' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,974 [289894] barman.server DEBUG: Starting check: 'pg_receivexlog'
2023-06-07 09:10:12,974 [289894] barman.server DEBUG: Check 'pg_receivexlog' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,974 [289894] barman.server DEBUG: Starting check: 'pg_receivexlog compatible'
2023-06-07 09:10:12,974 [289894] barman.server DEBUG: Check 'pg_receivexlog compatible' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,974 [289894] barman.server DEBUG: Starting check: 'receive-wal running'
2023-06-07 09:10:12,975 [289894] barman.server DEBUG: Check 'receive-wal running' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,975 [289894] barman.server DEBUG: Starting check: 'archiver errors'
2023-06-07 09:10:12,975 [289894] barman.server DEBUG: Check 'archiver errors' succeeded for server 'pg2023-05-31'
2023-06-07 09:10:12,976 [289894] barman.backup DEBUG: initialising backup information
2023-06-07 09:10:12,990 [289894] barman.backup INFO: Starting backup using postgres method for server pg2023-05-31 in /var/lib/barman/pg2023-05-31/base/20230607T091012
2023-06-07 09:10:12,992 [289894] barman.backup_executor DEBUG: detecting data directory
2023-06-07 09:10:12,999 [289894] barman.backup_executor DEBUG: detecting tablespaces
2023-06-07 09:10:13,000 [289894] barman.backup_executor DEBUG: issuing start backup command
2023-06-07 09:10:13,015 [289894] barman.backup_executor INFO: Backup start at LSN: 0/25000060 (000000010000000000000025, 00000060)
2023-06-07 09:10:13,016 [289894] barman.backup_executor INFO: Starting backup copy via pg_basebackup for 20230607T091012
2023-06-07 09:10:13,018 [289894] barman.command_wrappers DEBUG: Command: ['/usr/bin/pg_basebackup', '--dbname=dbname=replication host=192.168.121.28 options=-cdatestyle=iso replication=true user=streaming_barman application_name=barman_streaming_backup', '-v', '--no-password', '--pgdata=/var/lib/barman/pg2023-05-31/base/20230607T091012/data', '--no-slot', '--wal-method=none']
2023-06-07 09:10:13,345 [261983] barman.command_wrappers INFO: pg2023-05-31: pg_receivewal: finished segment at 0/26000000 (timeline 1)
2023-06-07 09:10:13,980 [261983] barman.command_wrappers INFO: pg2023-05-31: pg_receivewal: finished segment at 0/27000000 (timeline 1)

What I'm doing wrong? It's backup supposed to be run on background in order to work?

Why logs seems to be OK but operation seems hung?

So many questions!

Thanks a lot for your time

[Martin Marques]

Hi,

On Wed, 7 Jun 2023 at 09:15, Tan Mientras <tanimi...@gmail.com> wrote:
>
>
> barman backup pg2023-05-31
> Starting backup using postgres method for server pg2023-05-31 in /var/lib/barman/pg2023-05-31/base/20230607T091012
> Backup start at LSN: 0/25000060 (000000010000000000000025, 00000060)
> Starting backup copy via pg_basebackup for 20230607T091012
> HANGS HERE
>
> /var/log/barman/barman.log
> 2023-06-07 09:10:12,706 [289894] barman.config DEBUG: Including configuration file: pg.conf
> 2023-06-07 09:10:12,708 [289894] barman.cli DEBUG: Initialised Barman version 2.18 (config: /etc/barman.conf, args: {'color': 'auto', 'quiet': False, 'debug': False, 'format': 'console', 'command': 'backup', 'server_name': ['pg2023-05-31'], 'reuse_backup': None, 'retry_times': None, 'retry_sleep': None, 'jobs': None, 'wait': False, 'wait_timeout': None, 'func': <function backup at 0x7f2683153370>})
> 2023-06-07 09:10:12,734 [289894] barman.backup_executor DEBUG: The default backup strategy for postgres backup_method is: concurrent_backup

[snip]

> 2023-06-07 09:10:12,976 [289894] barman.backup DEBUG: initialising backup information
> 2023-06-07 09:10:12,990 [289894] barman.backup INFO: Starting backup using postgres method for server pg2023-05-31 in /var/lib/barman/pg2023-05-31/base/20230607T091012
> 2023-06-07 09:10:12,992 [289894] barman.backup_executor DEBUG: detecting data directory
> 2023-06-07 09:10:12,999 [289894] barman.backup_executor DEBUG: detecting tablespaces
> 2023-06-07 09:10:13,000 [289894] barman.backup_executor DEBUG: issuing start backup command
> 2023-06-07 09:10:13,015 [289894] barman.backup_executor INFO: Backup start at LSN: 0/25000060 (000000010000000000000025, 00000060)
> 2023-06-07 09:10:13,016 [289894] barman.backup_executor INFO: Starting backup copy via pg_basebackup for 20230607T091012
> 2023-06-07 09:10:13,018 [289894] barman.command_wrappers DEBUG: Command: ['/usr/bin/pg_basebackup', '--dbname=dbname=replication host=192.168.121.28 options=-cdatestyle=iso replication=true user=streaming_barman application_name=barman_streaming_backup', '-v', '--no-password', '--pgdata=/var/lib/barman/pg2023-05-31/base/20230607T091012/data', '--no-slot', '--wal-method=none']
> 2023-06-07 09:10:13,345 [261983] barman.command_wrappers INFO: pg2023-05-31: pg_receivewal: finished segment at 0/26000000 (timeline 1)
> 2023-06-07 09:10:13,980 [261983] barman.command_wrappers INFO: pg2023-05-31: pg_receivewal: finished segment at 0/27000000 (timeline 1)

How big is the database you are backing up? Is there any activity
going on? Network, CPU, disk I/O?

Did you check for any errors on the postgres logs?

[Mike Wallace]

In addition to Martin's suggestions, it's also worth upgrading from
Barman 2.18 to version 3.5.0 - from version 3.1.0 onwards the output
from `pg_basebackup` is logged to the Barman log file while the
command is running (in older versions the output is not logged until
the command completes, which makes it difficult to debug cases like
this where the command is not completing).

> --
> --
> You received this message because you are subscribed to the "Barman for PostgreSQL" group.
> To post to this group, send email to pgba...@googlegroups.com
> To unsubscribe from this group, send email to
> pgbarman+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/pgbarman?hl=en?hl=en-GB
>
> ---
> You received this message because you are subscribed to the Google Groups "Barman, Backup and Recovery Manager for PostgreSQL" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pgbarman+u...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/pgbarman/CAJGXRGqW%2B%2Bqf3AdUaStmeER1oO0mj8mDavF719hphUo61feeLQ%40mail.gmail.com.

[Martin Marques]

Oh dear! I hadn't noticed the version of barman until now. Yes, that
is just way too old. Upgrade barman, and try again.

On Wed, 7 Jun 2023 at 10:52, 'Mike Wallace' via Barman, Backup and

> To view this discussion on the web, visit https://groups.google.com/d/msgid/pgbarman/CAGxDL9%2BHcfoqW9PmakeejdFzuMP46QuG_tcE677doJQnGUoWxg%40mail.gmail.com.

[Tan Mientras]

I'm going to re-create my test environment using most recent barman version.

Thanks a lot.

[Tan Mientras]

After reinstalling the environment on a Ubuntu 22.04, this is what I'm getting:

barman@vmbarmantest:~$ barman --version
3.4.0 Barman by EnterpriseDB (www.enterprisedb.com)

barman@vmbarmantest:~$ barman backup pg2023-06-08
Starting backup using postgres method for server pg2023-06-08 in /var/lib/barman/pg2023-06-08/base/20230608T092809
Backup start at LSN: 0/29000060 (000000010000000000000029, 00000060)
Starting backup copy via pg_basebackup for 20230608T092809

HANGS HERE

barman@vmbarmantest:~$ sudo tail /var/log/barman/barman.log

2023-06-08 09:28:09,664 [56028] barman.config DEBUG: Including configuration file: pg.conf
2023-06-08 09:28:09,665 [56028] barman.cli DEBUG: Initialised Barman version 3.4.0 (config: /etc/barman.conf, args: {'color': 'auto', 'quiet': False, 'debug': False, 'format': 'console', 'command': 'backup', 'server_name': ['pg2023-06-08'], 'reuse_backup': None, 'retry_times': None, 'retry_sleep': None, 'jobs': None, 'wait': False, 'wait_timeout': None, 'backup_name': None, 'func': <function backup at 0x7ff2955f0790>})
2023-06-08 09:28:09,679 [56028] barman.backup_executor DEBUG: The default backup strategy for postgres backup_method is: concurrent_backup
2023-06-08 09:28:09,680 [56028] barman.server DEBUG: Starting check: 'WAL archive'
2023-06-08 09:28:09,680 [56028] barman.server DEBUG: Starting check: 'empty incoming directory'
2023-06-08 09:28:09,680 [56028] barman.server DEBUG: Starting check: 'empty streaming directory'
2023-06-08 09:28:09,680 [56028] barman.server DEBUG: Starting check: 'PostgreSQL'
2023-06-08 09:28:09,759 [56028] barman.command_wrappers DEBUG: Command: ['/usr/bin/pg_receivewal', '--version']
2023-06-08 09:28:09,802 [56028] Command DEBUG: pg_receivewal (PostgreSQL) 14.8 (Ubuntu 14.8-0ubuntu0.22.04.1)
2023-06-08 09:28:09,802 [56028] barman.command_wrappers DEBUG: Command return code: 0
2023-06-08 09:28:09,803 [56028] barman.command_wrappers DEBUG: Command stdout: pg_receivewal (PostgreSQL) 14.8 (Ubuntu 14.8-0ubuntu0.22.04.1)

2023-06-08 09:28:09,803 [56028] barman.command_wrappers DEBUG: Command stderr:
2023-06-08 09:28:09,805 [56028] barman.wal_archiver DEBUG: Look for 'barman_receive_wal' in 'synchronous_standby_names': ['']
2023-06-08 09:28:09,805 [56028] barman.wal_archiver DEBUG: Synchronous WAL streaming for barman_receive_wal: False
2023-06-08 09:28:09,806 [56028] barman.command_wrappers DEBUG: Command: ['/usr/bin/pg_basebackup', '--version']
2023-06-08 09:28:09,843 [56028] Command DEBUG: pg_basebackup (PostgreSQL) 14.8 (Ubuntu 14.8-0ubuntu0.22.04.1)
2023-06-08 09:28:09,844 [56028] barman.command_wrappers DEBUG: Command return code: 0
2023-06-08 09:28:09,844 [56028] barman.command_wrappers DEBUG: Command stdout: pg_basebackup (PostgreSQL) 14.8 (Ubuntu 14.8-0ubuntu0.22.04.1)

2023-06-08 09:28:09,844 [56028] barman.command_wrappers DEBUG: Command stderr:
2023-06-08 09:28:09,845 [56028] barman.server DEBUG: Check 'PostgreSQL' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,845 [56028] barman.server DEBUG: Starting check: 'superuser or standard user with backup privileges'
2023-06-08 09:28:09,845 [56028] barman.server DEBUG: Check 'superuser or standard user with backup privileges' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,845 [56028] barman.server DEBUG: Starting check: 'PostgreSQL streaming'
2023-06-08 09:28:09,845 [56028] barman.server DEBUG: Check 'PostgreSQL streaming' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,845 [56028] barman.server DEBUG: Starting check: 'wal_level'
2023-06-08 09:28:09,845 [56028] barman.server DEBUG: Check 'wal_level' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,845 [56028] barman.server DEBUG: Starting check: 'replication slot'
2023-06-08 09:28:09,846 [56028] barman.server DEBUG: Check 'replication slot' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,846 [56028] barman.server DEBUG: Starting check: 'directories'
2023-06-08 09:28:09,846 [56028] barman.server DEBUG: Check 'directories' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,846 [56028] barman.server DEBUG: Starting check: 'retention policy settings'
2023-06-08 09:28:09,846 [56028] barman.server DEBUG: Check 'retention policy settings' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,846 [56028] barman.server DEBUG: Starting check: 'backup maximum age'
2023-06-08 09:28:09,846 [56028] barman.server DEBUG: Check 'backup maximum age' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,846 [56028] barman.server DEBUG: Starting check: 'backup minimum size'
2023-06-08 09:28:09,846 [56028] barman.server DEBUG: Check 'backup minimum size' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,846 [56028] barman.server DEBUG: Starting check: 'wal maximum age'
2023-06-08 09:28:09,846 [56028] barman.server DEBUG: Check 'wal maximum age' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,846 [56028] barman.server DEBUG: Starting check: 'wal size'
2023-06-08 09:28:09,846 [56028] barman.server DEBUG: Check 'wal size' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,846 [56028] barman.server DEBUG: Starting check: 'compression settings'
2023-06-08 09:28:09,847 [56028] barman.server DEBUG: Check 'compression settings' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,847 [56028] barman.server DEBUG: Starting check: 'failed backups'
2023-06-08 09:28:09,847 [56028] barman.server DEBUG: Check 'failed backups' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,847 [56028] barman.server DEBUG: Starting check: 'minimum redundancy requirements'
2023-06-08 09:28:09,847 [56028] barman.server DEBUG: Check 'minimum redundancy requirements' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,847 [56028] barman.server DEBUG: Starting check: 'pg_basebackup'
2023-06-08 09:28:09,847 [56028] barman.server DEBUG: Check 'pg_basebackup' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,847 [56028] barman.server DEBUG: Starting check: 'pg_basebackup compatible'
2023-06-08 09:28:09,847 [56028] barman.server DEBUG: Check 'pg_basebackup compatible' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,848 [56028] barman.server DEBUG: Starting check: 'pg_basebackup supports tablespaces mapping'
2023-06-08 09:28:09,849 [56028] barman.server DEBUG: Check 'pg_basebackup supports tablespaces mapping' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,849 [56028] barman.server DEBUG: Starting check: 'configuration'
2023-06-08 09:28:09,849 [56028] barman.server DEBUG: Starting check: 'systemid coherence'
2023-06-08 09:28:09,849 [56028] barman.server DEBUG: Check 'systemid coherence' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,850 [56028] barman.server DEBUG: Starting check: 'pg_receivexlog'
2023-06-08 09:28:09,850 [56028] barman.server DEBUG: Check 'pg_receivexlog' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,850 [56028] barman.server DEBUG: Starting check: 'pg_receivexlog compatible'
2023-06-08 09:28:09,850 [56028] barman.server DEBUG: Check 'pg_receivexlog compatible' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,850 [56028] barman.server DEBUG: Starting check: 'receive-wal running'
2023-06-08 09:28:09,850 [56028] barman.server DEBUG: Check 'receive-wal running' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,850 [56028] barman.server DEBUG: Starting check: 'archiver errors'
2023-06-08 09:28:09,850 [56028] barman.server DEBUG: Check 'archiver errors' succeeded for server 'pg2023-06-08'
2023-06-08 09:28:09,852 [56028] barman.backup DEBUG: initialising backup information
2023-06-08 09:28:09,857 [56028] barman.backup INFO: Starting backup using postgres method for server pg2023-06-08 in /var/lib/barman/pg2023-06-08/base/20230608T092809
2023-06-08 09:28:09,858 [56028] barman.backup_executor DEBUG: detecting data directory
2023-06-08 09:28:09,862 [56028] barman.backup_executor DEBUG: detecting tablespaces
2023-06-08 09:28:09,863 [56028] barman.backup_executor DEBUG: issuing start backup command
2023-06-08 09:28:09,867 [56028] barman.backup_executor INFO: Backup start at LSN: 0/29000060 (000000010000000000000029, 00000060)
2023-06-08 09:28:09,868 [56028] barman.backup_executor INFO: Starting backup copy via pg_basebackup for 20230608T092809
2023-06-08 09:28:09,868 [56028] barman.command_wrappers DEBUG: Command: ['/usr/bin/pg_basebackup', '--dbname=dbname=replication host=192.168.121.28 options=-cdatestyle=iso replication=true user=streaming_barman application_name=barman_streaming_backup', '-v', '--no-password', '--pgdata=/var/lib/barman/pg2023-06-08/base/20230608T092809/data', '--no-slot', '--wal-method=none']
2023-06-08 09:28:09,935 [56028] barman.backup_executor INFO: pg_basebackup: initiating base backup, waiting for checkpoint to complete
2023-06-08 09:28:10,124 [17737] barman.command_wrappers INFO: pg2023-06-08: pg_receivewal: finished segment at 0/2A000000 (timeline 1)
2023-06-08 09:28:10,173 [56028] barman.backup_executor INFO: pg_basebackup: checkpoint completed
2023-06-08 09:28:10,455 [17737] barman.command_wrappers INFO: pg2023-06-08: pg_receivewal: finished segment at 0/2B000000 (timeline 1)
2023-06-08 09:28:16,377 [56028] barman.backup_executor INFO: NOTICE:  base backup done, waiting for required WAL segments to be archived

You received this message because you are subscribed to a topic in the Google Groups "Barman, Backup and Recovery Manager for PostgreSQL" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pgbarman/XKGF8LoEu98/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pgbarman+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pgbarman/33519186-e6b3-4483-8e75-1ac2e493f96an%40googlegroups.com.

[Tan Mientras]

...
2023-06-08 09:35:49,655 [56168] barman.backup_executor INFO: WARNING:  still waiting for all required WAL segments to be archived (120 seconds elapsed)
2023-06-08 09:35:49,656 [56168] barman.backup_executor INFO: HINT:  Check that your archive_command is executing properly.  You can safely cancel this backup, but the database backup will not be usable without all the WAL segments.
...

[Luca Ferrari]

On Thu, Jun 8, 2023 at 9:36 AM Tan Mientras <tanimi...@gmail.com> wrote:
>
> ...
> 2023-06-08 09:35:49,655 [56168] barman.backup_executor INFO: WARNING: still waiting for all required WAL segments to be archived (120 seconds elapsed)
> 2023-06-08 09:35:49,656 [56168] barman.backup_executor INFO: HINT: Check that your archive_command is executing properly. You can safely cancel this backup, but the database backup will not be usable without all the WAL segments.
> ...

Check the PostgreSQL logs, you probably will find out that the
archive_command is not running properly.
Become the postgres system user, run the command manually and diagnose
the failure.

Luca

[Tan Mientras]

considering I'm not using rsync (just streaming mode), which is the archive_command to test on postgres?

pg_basebackup ?

pg_receivewal ?

barman-wal-archive ?

[Luca Ferrari]

On Thu, Jun 8, 2023 at 2:15 PM Tan Mientras <tanimi...@gmail.com> wrote:
>
> considering I'm not using rsync (just streaming mode), which is the archive_command to test on postgres?
> pg_basebackup ?
> pg_receivewal ?
> barman-wal-archive ?

If you are using streaming only, as your initial configuration was
doing, you don't need an archive_command at all. My suspect is that
barman is expecting PostgreSQL to send the WALs, instead of streaming.
I would double check archive_command and the PostgreSQL logs to see if
there's an hint.

Luca

[Mike Wallace]

Hi Tan,

The following log messages are coming from pg_basebackup:

2023-06-08 09:28:16,377 [56028] barman.backup_executor INFO: NOTICE:  base backup done, waiting for required WAL segments to be archived

2023-06-08 09:35:49,655 [56168] barman.backup_executor INFO: WARNING:  still waiting for all required WAL segments to be archived (120 seconds elapsed)

2023-06-08 09:35:49,656 [56168] barman.backup_executor INFO: HINT:  Check that your archive_command is executing properly.  You can safely cancel this backup, but the database backup will not be usable without all the WAL segments.

This tells us that pg_basebackup is finishing thre copy phase of the backup and then waiting for a final WAL switch and archive to complete - this never happens and therefore the process continues to wait.

As Luca suggests, the most likely reason for this WAL switch failing is the value of `archive_command` in your PostgreSQL configuration.

Even though Barman is configured with `archiver = off`, this has no influence on the behaviour of PostgreSQL - if the PostgreSQL configuration contains an `archive_command` value and `archive_mode = on` is also set in the PostgreSQL configuration, then PostgreSQL is going to run `archive_command` every time there is a WAL switch. If the `archive_command` related to that WAL switch does not succeed then it will prevent backups from completing.

Since you are relying on WAL streaming, you should check the value of `archive_mode` in your PostgreSQL config file - if you have `archive_mode = on`, change it to `archive_mode = off`. You will then need to restart your PostgreSQL server.

Hope this helps,

Mike

> --
> --
> You received this message because you are subscribed to the "Barman for PostgreSQL" group.
> To post to this group, send email to pgba...@googlegroups.com
> To unsubscribe from this group, send email to
> pgbarman+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/pgbarman?hl=en?hl=en-GB
>
> ---
> You received this message because you are subscribed to the Google Groups "Barman, Backup and Recovery Manager for PostgreSQL" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pgbarman+u...@googlegroups.com.

> To view this discussion on the web, visit https://groups.google.com/d/msgid/pgbarman/CAKoxK%2B42Mmm2YRZ9ZkW%3DFYNW58dzLUHpcZdCp2mWpYeVfM%3DwWQ%40mail.gmail.com.

[Luca Ferrari]

On Thu, Jun 8, 2023 at 5:03 PM 'Mike Wallace' via Barman, Backup and
Recovery Manager for PostgreSQL <pgba...@googlegroups.com> wrote:
> Since you are relying on WAL streaming, you should check the value of `archive_mode` in your PostgreSQL config file - if you have `archive_mode = on`, change it to `archive_mode = off`. You will then need to restart your PostgreSQL server.

I usually leave archive_mode to on, and change archive_command to a
fake always true command, so I don't need a PostgreSQL restart.

Luca

[Martin Marques]

Hello,

Do you have `barman cron` running? Are the WALs that need to be
archived in $barman_home/streaming/ or where ever you have configured
wal streaming to put the WALs? If the WALs are still there, and they
should be there from the logs sent earlier, it means you are not
running `barman archive-wal` (which is executed by `barman cron`).

See if that fixes your problem.

Kind regards, Martín

> To view this discussion on the web, visit https://groups.google.com/d/msgid/pgbarman/CA%2BTYS86V-EXHfaKSGWqDasmcrFnL%3DPHemQgDXBPqvr68TepaDw%40mail.gmail.com.

[Tan Mientras]

On Thursday, June 8, 2023 at 5:03:53 PM UTC+2 Mike Wallace wrote:

This tells us that pg_basebackup is finishing thre copy phase of the backup and then waiting for a final WAL switch and archive to complete - this never happens and therefore the process continues to wait.

As Luca suggests, the most likely reason for this WAL switch failing is the value of `archive_command` in your PostgreSQL configuration.

Even though Barman is configured with `archiver = off`, this has no influence on the behaviour of PostgreSQL - if the PostgreSQL configuration contains an `archive_command` value and `archive_mode = on` is also set in the PostgreSQL configuration, then PostgreSQL is going to run `archive_command` every time there is a WAL switch. If the `archive_command` related to that WAL switch does not succeed then it will prevent backups from completing.

Since you are relying on WAL streaming, you should check the value of `archive_mode` in your PostgreSQL config file - if you have `archive_mode = on`, change it to `archive_mode = off`. You will then need to restart your PostgreSQL server.

Indeed, archive_mode was set to on in PostgreSQL. After setting it to off,  backup command ended without hesitation!

Thanks a lot again for your kind help!