Re: [pgbarman] Restore Fail

138 views
Skip to first unread message
Message has been deleted

Luca Ferrari

unread,
Feb 14, 2022, 4:18:51 AM2/14/22
to Barman, Backup and Recovery Manager for PostgreSQL
On Mon, Feb 14, 2022 at 8:34 AM Jose Manuel Valseca Echevarria
<josemanue...@euigs.com> wrote:
>
>
> < 2022-02-14 08:28:34.256 CET > LOG: starting archive recovery
> < 2022-02-14 08:28:34.257 CET > LOG: invalid primary checkpoint record
> < 2022-02-14 08:28:34.257 CET > LOG: invalid secondary checkpoint record

Either the backup is corrupted, or the WALs have not been correctly
shipped to the restore machine.


> a question. What is the purpose of the recovery.conf file?

It tells PostgreSQL that it should not try a "normal" start, rather a
crash recovery pulling WALs from a specific place.
And it tells us that you are using PostgreSQL less than version 12.

Is barman providing any hint about the backup? Is it in good helath?
Have you tried again doing the restore?

Luca
Message has been deleted
Message has been deleted

Luca Ferrari

unread,
Feb 15, 2022, 3:38:42 AM2/15/22
to Barman, Backup and Recovery Manager for PostgreSQL
On Mon, Feb 14, 2022 at 8:53 PM Jose Manuel Valseca Echevarria
<josemanue...@euigs.com> wrote:
>
> I have launched two restores from the copy of 02/12 and both have failed. Today I launched the restore of a new copy and it also failed..
>

The problem seems always the same: the WALs are corrupted.
Either your restore to the target machine is not working (e.g., rsync
is misbehaving) or, most probably, your backups are not working as
expected.

Assuming your database is working now, that is not clear at all, can
you take a fresh backup and restore that?
In the case your database is not working, can you restore a previous
backup to see if there's any working?

Luca
Message has been deleted

Luca Ferrari

unread,
Feb 15, 2022, 5:47:59 AM2/15/22
to Barman, Backup and Recovery Manager for PostgreSQL
On Tue, Feb 15, 2022 at 11:12 AM Jose Manuel Valseca Echevarria
<josemanue...@euigs.com> wrote:
> WARNING: 'get-wal' is in the specified 'recovery_options'.

I don't think there is a problem in getting the wal, but can you run a
`barman get-wal` command as the PostgreSQL user?


> How can i know if my bd is correct?

Checking the data inside it.

> How can i check that my backup is correct?

Barman tells you the backup is sane, and seem to be your situation.

> How can I check that the transfer has been successful? When it is finished it tells me that everything has gone successfully.

Yes, and since you have get-wal there is no wal shipping during the
recovery phase, therefore the transfer should always be correct.
Apart from starting over on a new clean location (PGDATA), I have no ideas.

Luca
Message has been deleted

Michael Wallace

unread,
Feb 15, 2022, 6:33:55 AM2/15/22
to pgba...@googlegroups.com
Take a look in the recovery.conf file for the `restore_command` setting - since you are recovering using the `--remote-ssh-command` option this should be a `barman-wal-restore` command. It should be possible to run the full command successfully as the postgres user. You will need to provide values for %f (the name of the WAL to retrieve) and %p (the destination path to which the WAL should be copied) when testing this. See https://www.postgresql.org/docs/9.6/archive-recovery-settings.html for more information about restore_command.

If this command does not complete successfully then PostgreSQL will not be able to retrieve the necessary WALs for recovery and this would explain the problem. If this command does complete successfully then unfortunately the problem is something harder to diagnose.

On Tue, Feb 15, 2022 at 10:55 AM Jose Manuel Valseca Echevarria <josemanue...@euigs.com> wrote:
I don't quite understand the get-wal command. I don't know how to use it :(

Sorry

--
--
You received this message because you are subscribed to the "Barman for PostgreSQL" group.
To post to this group, send email to pgba...@googlegroups.com
To unsubscribe from this group, send email to
pgbarman+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/pgbarman?hl=en?hl=en-GB

---
You received this message because you are subscribed to the Google Groups "Barman, Backup and Recovery Manager for PostgreSQL" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pgbarman+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pgbarman/CAKoxK%2B58mpD%2BKs2%3DORhm5m2qUbSELxWBP4Fi4WN5fSYFrSPUsg%40mail.gmail.com.


--

Jose Manuel Valseca                            

 Asset & Recovery

 EUIGS - Admiral Group

#LeadingTheFuture #BusinessLover

Awesome Job? 

--
--
You received this message because you are subscribed to the "Barman for PostgreSQL" group.
To post to this group, send email to pgba...@googlegroups.com
To unsubscribe from this group, send email to
pgbarman+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/pgbarman?hl=en?hl=en-GB

---
You received this message because you are subscribed to the Google Groups "Barman, Backup and Recovery Manager for PostgreSQL" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pgbarman+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pgbarman/CAEDwTouRzQYS25HLbm7rNW1Yi-4%2BVi1_Lpr8cYiwtegB-hBg3w%40mail.gmail.com.
Message has been deleted

Michael Wallace

unread,
Feb 16, 2022, 4:28:45 AM2/16/22
to pgba...@googlegroups.com
Good morning Jose,

The destination path should include filename, e.g. `/opt/pgsql/9.6/data/pg_wal/00000001000014580000008D`.

Best,

Michael

On Wed, Feb 16, 2022 at 7:46 AM Jose Manuel Valseca Echevarria <josemanue...@euigs.com> wrote:
Good morning. I have been checking what Michael told me and I get the following:
-bash-4.2$ ssh barman@XXXXXXX
Last login: Wed Feb 16 08:27:01 2022 from 192.168.54.131
The connection of the postgres user to the barman server via ssh is successful.

With these data:
cat backup.info
backup_label='START WAL LOCATION: 1457/B5827670 (file 0000000100001457000000B5)\nCHECKPOINT LOCATION: 1457/B872D1A8\nBACKUP METHOD: streamed\nBACKUP FROM: master\nSTART TIME: 2022-02-15 09:33:06 CET\nLABEL: Barman backup alfresco 20220215T092847\n'
begin_offset=8550000
begin_time=2022-02-15 09:28:48.105056+01:00
begin_wal=0000000100001457000000B5
begin_xlog=1457/B5827670
config_file=/opt/pgsql/9.6/data/postgresql.conf
copy_stats={'total_time': 11858.943803, 'number_of_workers': 1, 'serialized_copy_time': 11849.630775, 'serialized_copy_time_per_item': {'pgdata': 11849.430592, 'pg_control': 0.200183}, 'copy_time_per_item': {'pgdata': 11849.577764, 'pg_control': 0.200183}, 'copy_time'
: 11849.791569, 'analysis_time_per_item': {'pgdata': 8.810502}, 'analysis_time': 8.810502}
deduplicated_size=3377939345606
end_offset=7237480
end_time=2022-02-15 12:50:45.145956+01:00
end_wal=00000001000014580000008D
end_xlog=1458/8D6E6F68
error=None
hba_file=/opt/pgsql/9.6/data/pg_hba.conf
ident_file=/opt/pgsql/9.6/data/pg_ident.conf
included_files=None
mode=rsync-concurrent
pgdata=/opt/pgsql/9.6/data
server_name=alfresco
size=3377939345606
status=DONE
systemid=6386591195023846612
tablespaces=None
timeline=1
version=90602
xlog_segment_size=16777216

using the following command get the following output:
-bash-4.2$ barman-wal-restore -P -U barman barman.XXXXX alfresco 00000001000014580000008D /opt/pgsql/9.6/data
ERROR: WAL_DEST cannot be a directory: /opt/pgsql/9.6/data

Where am I going wrong?

thanks





--

Jose Manuel Valseca                            

 Asset & Recovery

 EUIGS - Admiral Group

#LeadingTheFuture #BusinessLover

Awesome Job? 

--
--
You received this message because you are subscribed to the "Barman for PostgreSQL" group.
To post to this group, send email to pgba...@googlegroups.com
To unsubscribe from this group, send email to
pgbarman+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/pgbarman?hl=en?hl=en-GB

---
You received this message because you are subscribed to the Google Groups "Barman, Backup and Recovery Manager for PostgreSQL" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pgbarman+u...@googlegroups.com.
Message has been deleted

Luca Ferrari

unread,
Feb 16, 2022, 4:50:39 AM2/16/22
to Barman, Backup and Recovery Manager for PostgreSQL
On Wed, Feb 16, 2022 at 10:40 AM Jose Manuel Valseca Echevarria
<josemanue...@euigs.com> wrote:
>
> But I don't have a pg_wall directory on my server, what would be the correct path?

pg_xlog is the same of pg_wal for PostgreSQL versions priori to 10.

Luca
Message has been deleted
Message has been deleted

Michael Wallace

unread,
Feb 18, 2022, 5:51:14 AM2/18/22
to pgba...@googlegroups.com
Hi Jose,

Currently it isn't clear whether PostgreSQL is able to retrieve the WALs necessary for restoring the backup - did you have any success manually testing the barman-wal-restore command?

A few other questions:

* What is the value of archive_command in postgresql.conf?
* Can you check the contents of ${BARMAN_HOME}/alfresco/wals - there should be an xlog.db file and directories containing the actual WAL files.
* Are there any errors in the postgres logs (other than those you have already posted here) when it fails to start after recovery?
* Can you run `barman diagnose` and paste the output here?

Best,

Mike

On Fri, Feb 18, 2022 at 8:59 AM Jose Manuel Valseca Echevarria <josemanue...@euigs.com> wrote:
For information, on the production machine, the pg_xlog directory is a symbolic link to another path. I don't know if that is influencing it.
drwx------+ 19 postgres postgres 4096 Feb 18 08:24 .
drwxr-x---+ 4 postgres postgres 31 Feb 14 2017 ..
drwx------ 7 postgres postgres 66 Aug 3 2017 base
-rwx------ 1 postgres postgres 1138028544 Jan 27 09:24 core.41050
drwx------ 2 postgres postgres 4096 Feb 18 09:39 global
drwx------ 2 postgres postgres 4096 Feb 17 21:25 pg_clog
drwx------ 2 postgres postgres 6 Feb 13 2017 pg_commit_ts
drwx------ 2 postgres postgres 6 Feb 13 2017 pg_dynshmem
-rw------- 1 postgres postgres 5459 Dec 15 16:22 pg_hba.conf
-rw------- 1 postgres postgres 1636 Feb 13 2017 pg_ident.conf
drwx------ 2 postgres postgres 4096 Feb 1 11:30 pg_log
drwx------ 4 postgres postgres 37 Feb 13 2017 pg_logical
drwx------ 4 postgres postgres 34 Feb 13 2017 pg_multixact
drwx------ 2 postgres postgres 17 Feb 18 08:24 pg_notify
drwx------ 2 postgres postgres 6 Feb 13 2017 pg_replslot
drwx------ 2 postgres postgres 6 Feb 13 2017 pg_serial
drwx------ 2 postgres postgres 6 Feb 13 2017 pg_snapshots
drwx------ 2 postgres postgres 6 Feb 18 08:24 pg_stat
drwx------ 2 postgres postgres 80 Feb 18 09:58 pg_stat_tmp
drwx------ 2 postgres postgres 6 Feb 18 09:57 pg_subtrans
drwx------ 2 postgres postgres 6 Feb 13 2017 pg_tblspc
drwx------ 2 postgres postgres 6 Feb 13 2017 pg_twophase
-rw------- 1 postgres postgres 4 Feb 13 2017 PG_VERSION
lrwxrwxrwx 1 postgres postgres 22 Feb 13 2017 pg_xlog -> /opt/pgsql/9.6/pg_xlog
-rw------- 1 postgres postgres 88 Feb 13 2017 postgresql.auto.conf
-rw------- 1 postgres postgres 22367 Jul 29 2021 postgresql.conf
-rw------- 1 postgres postgres 56 Feb 18 08:24 postmaster.opts
-rw------- 1 postgres postgres 96 Feb 18 08:24 postmaster.pid

El vie, 18 feb 2022 a las 9:52, Jose Manuel Valseca Echevarria (<josemanue...@euigs.com>) escribió:
Good morning.  I have changed the strategy and cloned the production server. Once cloned I dumped the most recent backup and I still have the same problem. The database seems to be fine. It boots without problems and the logs are correct. I don't know what to check or what to look at, do you have any ideas, can I give you some information, I'm afraid it's a barman bug as it happened recently with the logs.

Best regards



--
--
You received this message because you are subscribed to the "Barman for PostgreSQL" group.
To post to this group, send email to pgba...@googlegroups.com
To unsubscribe from this group, send email to
pgbarman+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/pgbarman?hl=en?hl=en-GB

---
You received this message because you are subscribed to the Google Groups "Barman, Backup and Recovery Manager for PostgreSQL" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pgbarman+u...@googlegroups.com.
Message has been deleted

Michael Wallace

unread,
Feb 18, 2022, 12:19:56 PM2/18/22
to pgba...@googlegroups.com
Thanks Jose - I'm trying to create a similar setup using Postgres 9.6 to see if I can reproduce the problem.

Once thing which looks odd is that the server name has a space at the beginning in the diagnose output, e.g.:

   "servers": {
        " alfresco": {

This space is then being added to the various directories which Barman derives from the server name, e.g.:

                "incoming_wals_directory": "/var/lib/barman/ alfresco/incoming",

and:

                "wals_directory": "/var/lib/barman/ alfresco/wals"

Looking at the thread I'm wondering where this came from, since the earlier commands just reference the name "alfresco" without the leading space. Was this always present or did it get added recently? Either way, if there *is* a space at the beginning of the server name then it's possible Barman is not escaping it when it constructs the `restore_command` on recovery. For example, if I create a server named " main" locally then Barman will add the following restore command:

    restore_command = 'barman-wal-restore -P -U barman backup  main 000000010000000000000002 /tmp/restored_wal'

If I run this it will then fail because the leading space in front of main is neither quoted nor escaped, e.g..:

    > barman-wal-restore -P -U barman backup  main 000000010000000000000002 /tmp/restored_wal
    ERROR: Unknown server 'main'

If I manually add quotes around the server name then the command will succeed:

    > barman-wal-restore -P -U barman backup ' main' 000000010000000000000002 /tmp/restored_wal

None of this is relevant if the leading space in the server name " alfresco" was added just before you ran the diagnose command but I thought it was worth mentioning in case it yields some clues at your end.

On Fri, Feb 18, 2022 at 1:03 PM Jose Manuel Valseca Echevarria <josemanue...@euigs.com> wrote:
Hi Jose,

Currently it isn't clear whether PostgreSQL is able to retrieve the WALs necessary for restoring the backup - did you have any success manually testing the barman-wal-restore command?

A few other questions:

* What is the value of archive_command in postgresql.conf?
archive_command = 'rsync -a %p bar...@192.168.xx.xx:/backup/barman/alfresco/incoming/%f' # command to use to archive a logfile segment

* Can you check the contents of ${BARMAN_HOME}/alfresco/wals - there should be an xlog.db file and directories containing the actual WAL files.
[barman@barman01 wals]$ ll
total 720
drwxr-xr-x. 2 barman barman 12288 Feb 12 10:30 000000010000144D
drwxr-xr-x. 2 barman barman 12288 Feb 12 18:03 000000010000144E
drwxr-xr-x. 2 barman barman 12288 Feb 13 02:21 000000010000144F
drwxr-xr-x. 2 barman barman 12288 Feb 13 12:35 0000000100001450
drwxr-xr-x. 2 barman barman 12288 Feb 13 18:50 0000000100001451
drwxr-xr-x. 2 barman barman 12288 Feb 14 09:19 0000000100001452
drwxr-xr-x. 2 barman barman 12288 Feb 14 13:21 0000000100001453
drwxr-xr-x. 2 barman barman 12288 Feb 14 17:51 0000000100001454
drwxr-xr-x. 2 barman barman 12288 Feb 14 19:31 0000000100001455
drwxr-xr-x. 2 barman barman 12288 Feb 15 05:08 0000000100001456
drwxr-xr-x. 2 barman barman 12288 Feb 15 12:50 0000000100001457
drwxr-xr-x. 2 barman barman 12288 Feb 15 15:02 0000000100001458
drwxr-xr-x. 2 barman barman 12288 Feb 15 18:33 0000000100001459
drwxr-xr-x. 2 barman barman 12288 Feb 15 21:44 000000010000145A
drwxr-xr-x. 2 barman barman 12288 Feb 16 09:11 000000010000145B
drwxr-xr-x. 2 barman barman 12288 Feb 16 13:01 000000010000145C
drwxr-xr-x. 2 barman barman 12288 Feb 16 17:42 000000010000145D
drwxr-xr-x. 2 barman barman 12288 Feb 16 19:28 000000010000145E
drwxr-xr-x. 2 barman barman 12288 Feb 17 08:40 000000010000145F
drwxr-xr-x. 2 barman barman 12288 Feb 17 12:05 0000000100001460
drwxr-xr-x. 2 barman barman 12288 Feb 17 16:56 0000000100001461
drwxr-xr-x. 2 barman barman 12288 Feb 17 19:18 0000000100001462
drwxr-xr-x. 2 barman barman 12288 Feb 18 07:50 0000000100001463
drwxr-xr-x. 2 barman barman 12288 Feb 18 11:38 0000000100001464
drwxr-xr-x. 2 barman barman 8192 Feb 18 13:55 0000000100001465
-rw-r--r--. 1 barman barman 329156 Feb 18 13:55 xlog.db


* Are there any errors in the postgres logs (other than those you have already posted here) when it fails to start after recovery?
No other errors

* Can you run `barman diagnose` and paste the output here?
[barman@barman01 wals]$ barman diagnose
{
    "global": {
        "config": {
            "barman_home": "/var/lib/barman",
            "barman_user": "barman",
            "configuration_files_directory": "/opt/barman/etc",
            "errors_list": [],
            "log_file": "/opt/barman/log/barman.log",
            "log_level": "INFO"
        },
        "system_info": {
            "barman_ver": "2.17",
            "kernel_ver": "Linux barman01.XXXX.xxxxx.XXXXX 3.10.0-1160.45.1.el7.x86_64 #1 SMP Wed Oct 13 17:20:51 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux",
            "python_ver": "",
            "release": "LSB Version:\t:core-4.1-amd64:core-4.1-noarch\nDistributor ID:\tCentOS\nDescription:\tCentOS Linux release 7.9.2009 (Core)\nRelease:\t7.9.2009\nCodename:\tCore",
            "rsync_ver": "rsync  version 3.1.2  protocol version 31",
            "ssh_ver": "",
            "timestamp": "Fri Feb 18 13:58:08 2022"
        }
    },
    "servers": {
        " alfresco": {
            "backups": {
                "20220215T092847": {
                    "backup_id": "20220215T092847",
                    "backup_label": "'START WAL LOCATION: 1457/B5827670 (file 0000000100001457000000B5)\\nCHECKPOINT LOCATION: 1457/B872D1A8\\nBACKUP METHOD: streamed\\nBACKUP FROM: master\\nSTART TIME: 2022-02-15 09:33:06 CET\\nLABEL: Barman backup  alfresco 20220215T092847\\n'",
                    "begin_offset": 8550000,
                    "begin_time": "Tue Feb 15 09:28:48 2022",
                    "begin_wal": "0000000100001457000000B5",
                    "begin_xlog": "1457/B5827670",
                    "config_file": "/opt/pgsql/9.6/data/postgresql.conf",
                    "copy_stats": {
                        "analysis_time": 8.810502,
                        "analysis_time_per_item": {
                            "pgdata": 8.810502
                        },
                        "copy_time": 11849.791569,
                        "copy_time_per_item": {
                            "pg_control": 0.200183,
                            "pgdata": 11849.577764
                        },
                        "number_of_workers": 1,
                        "serialized_copy_time": 11849.630775,
                        "serialized_copy_time_per_item": {
                            "pg_control": 0.200183,
                            "pgdata": 11849.430592
                        },
                        "total_time": 11858.943803
                    },
                    "deduplicated_size": 3377939345606,
                    "end_offset": 7237480,
                    "end_time": "Tue Feb 15 12:50:45 2022",
                    "end_wal": "00000001000014580000008D",
                    "end_xlog": "1458/8D6E6F68",
                    "error": null,
                    "hba_file": "/opt/pgsql/9.6/data/pg_hba.conf",
                    "ident_file": "/opt/pgsql/9.6/data/pg_ident.conf",
                    "included_files": null,
                    "mode": "rsync-concurrent",
                    "pgdata": "/opt/pgsql/9.6/data",
                    "server_name": " alfresco",
                    "size": 3377939345606,
                    "status": "DONE",
                    "systemid": "6386591195023846612",
                    "tablespaces": null,
                    "timeline": 1,
                    "version": 90602,
                    "xlog_segment_size": 16777216
                }
            },
            "config": {
                "active": true,
                "archiver": true,
                "archiver_batch_size": 50,
                "backup_directory": "/var/lib/barman/ alfresco",
                "backup_method": "rsync",
                "backup_options": "concurrent_backup",
                "bandwidth_limit": null,
                "barman_home": "/var/lib/barman",
                "barman_lock_directory": "/var/lib/barman",
                "basebackup_retry_sleep": 30,
                "basebackup_retry_times": 0,
                "basebackups_directory": "/var/lib/barman/ alfresco/base",
                "check_timeout": 30,
                "compression": null,
                "conninfo": "host=192.168.54.10 user=barman dbname=postgres",
                "create_slot": "manual",
                "custom_compression_filter": null,
                "custom_compression_magic": null,
                "custom_decompression_filter": null,
                "description": " alfresco PostgreSQL Database (via SSH)",
                "disabled": false,
                "errors_directory": "/var/lib/barman/ alfresco/errors",
                "forward_config_path": false,
                "immediate_checkpoint": false,
                "incoming_wals_directory": "/var/lib/barman/ alfresco/incoming",
                "last_backup_maximum_age": null,
                "last_backup_minimum_size": null,
                "last_wal_maximum_age": null,
                "max_incoming_wals_queue": null,
                "minimum_redundancy": 0,
                "msg_list": [],
                "name": " alfresco",
                "network_compression": false,
                "parallel_jobs": 1,
                "path_prefix": "/usr/pgsql-9.6/bin",
                "post_archive_retry_script": null,
                "post_archive_script": null,
                "post_backup_retry_script": null,
                "post_backup_script": "/opt/barman/bin/backup_check.sh",
                "post_delete_retry_script": null,
                "post_delete_script": null,
                "post_recovery_retry_script": null,
                "post_recovery_script": null,
                "post_wal_delete_retry_script": null,
                "post_wal_delete_script": null,
                "pre_archive_retry_script": null,
                "pre_archive_script": null,
                "pre_backup_retry_script": null,
                "pre_backup_script": "/opt/barman/bin/backup_check.sh",
                "pre_delete_retry_script": null,
                "pre_delete_script": null,
                "pre_recovery_retry_script": null,
                "pre_recovery_script": null,
                "pre_wal_delete_retry_script": null,
                "pre_wal_delete_script": null,
                "primary_ssh_command": null,
                "recovery_options": "get-wal",
                "retention_policy": "redundancy 1 b",
                "retention_policy_mode": "auto",
                "reuse_backup": "link",
                "slot_name": null,
                "ssh_command": "ssh post...@192.168.54.10",
                "streaming_archiver": false,
                "streaming_archiver_batch_size": 0,
                "streaming_archiver_name": "barman_receive_wal",
                "streaming_backup_name": "barman_streaming_backup",
                "streaming_conninfo": "host=192.168.54.10 user=barman dbname=postgres",
                "streaming_wals_directory": "/var/lib/barman/ alfresco/streaming",
                "tablespace_bandwidth_limit": null,
                "wal_retention_policy": "simple-wal 1 b",
                "wals_directory": "/var/lib/barman/ alfresco/wals"
            },
            "status": {
                "archive_command": "rsync -a %p bar...@192.168.54.17:/backup/barman/ alfresco/incoming/%f",
                "archive_mode": "on",
                "archive_timeout": 0,
                "archived_count": 24503,
                "checkpoint_timeout": 300,
                "config_file": "/opt/pgsql/9.6/data/postgresql.conf",
                "current_archived_wals_per_second": 0.0128139310415201,
                "current_lsn": "1465/8073C488",
                "current_size": 3385711516502.0,
                "current_xlog": "000000010000146500000080",
                "data_checksums": "off",
                "data_directory": "/opt/pgsql/9.6/data",
                "failed_count": 346,
                "has_backup_privileges": true,
                "hba_file": "/opt/pgsql/9.6/data/pg_hba.conf",
                "hot_standby": "off",
                "ident_file": "/opt/pgsql/9.6/data/pg_ident.conf",
                "is_archiving": true,
                "is_in_recovery": false,
                "is_superuser": true,
                "last_archived_time": "Fri Feb 18 13:57:33 2022",
                "last_archived_wal": "00000001000014650000007F",
                "last_failed_time": "Thu Jan 27 12:12:24 2022",
                "last_failed_wal": "0000000100001405000000D2",
                "max_replication_slots": "0",
                "max_wal_senders": "6",
                "pgespresso_installed": false,
                "postgres_systemid": "6386591195023846612",
                "replication_slot": null,
                "replication_slot_support": true,
                "server_txt_version": "9.6.2",
                "stats_reset": "Thu Jan 27 10:47:55 2022",
                "synchronous_standby_names": [
                    ""
                ],
                "wal_compression": "off",
                "wal_keep_segments": "0",
                "wal_level": "replica",
                "xlog_segment_size": 16777216
            },
            "system_info": {
                "kernel_ver": "Linux  alfresco1.XXXX.xxxxx.XXXXX 3.10.0-1127.13.1.el7.x86_64 #1 SMP Tue Jun 23 15:46:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux",
                "python_ver": "",
                "release": "RedHat Linux CentOS Linux release 7.8.2003 (Core)",
                "rsync_ver": "rsync  version 3.1.2  protocol version 31",
                "ssh_ver": ""
            },
            "wals": {
                "last_archived_wal_per_timeline": {
                    "00000001": {
                        "compression": null,
                        "name": "00000001000014650000007F",
                        "size": 16777216,
                        "time": 1645189053.2366154
                    }
                }
            }
        },
        "dwh": {
            "backups": {
                "20220212T065724": {
                    "backup_id": "20220212T065724",
                    "backup_label": "'START WAL LOCATION: D4EE/B3C1C2A8 (file 000000010000D4EE000000B3)\\nCHECKPOINT LOCATION: D4EF/67D37338\\nBACKUP METHOD: streamed\\nBACKUP FROM: master\\nSTART TIME: 2022-02-12 06:00:48 GMT\\nLABEL: Barman backup dwh 20220212T065724\\n'",
                    "begin_offset": 12698280,
                    "begin_time": "Sat Feb 12 06:57:28 2022",
                    "begin_wal": "000000010000D4EE000000B3",
                    "begin_xlog": "D4EE/B3C1C2A8",
                    "config_file": "/opt/disk1/postgresql/9.6/data/postgresql.conf",
                    "copy_stats": {
                        "analysis_time": 9.309123,
                        "analysis_time_per_item": {
                            "pgdata": 4.541421,
                            "tbs_ac": 1.211323,
                            "tbs_dc": 1.090448,
                            "tbs_st": 2.465519
                        },
                        "copy_time": 26975.282342,
                        "copy_time_per_item": {
                            "pg_control": 0.524296,
                            "pgdata": 3538.264061,
                            "tbs_ac": 2178.24862,
                            "tbs_dc": 3425.73692,
                            "tbs_st": 17832.380772
                        },
                        "number_of_workers": 1,
                        "serialized_copy_time": 26974.749976,
                        "serialized_copy_time_per_item": {
                            "pg_control": 0.524296,
                            "pgdata": 3537.959399,
                            "tbs_ac": 2178.237037,
                            "tbs_dc": 3425.725401,
                            "tbs_st": 17832.303843
                        },
                        "total_time": 26984.672039
                    },
                    "deduplicated_size": 3020497756225,
                    "end_offset": 5450576,
                    "end_time": "Sat Feb 12 14:30:33 2022",
                    "end_wal": "000000010000D589000000AB",
                    "end_xlog": "D589/AB532B50",
                    "error": null,
                    "hba_file": "/opt/disk1/postgresql/9.6/data/pg_hba.conf",
                    "ident_file": "/opt/disk1/postgresql/9.6/data/pg_ident.conf",
                    "included_files": [
                        "/opt/disk1/postgresql/9.6/data/postgresql.auto.conf",
                        "/opt/disk1/postgresql/9.6/data/postgresql-backend_params.conf",
                        "/opt/disk1/postgresql/9.6/data/postgresql-ssl_params.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "pgdata": "/opt/disk1/postgresql/9.6/data",
                    "server_name": "dwh",
                    "size": 10773542804720,
                    "status": "DONE",
                    "systemid": "6410648610747580763",
                    "tablespaces": [
                        [
                            "tbs_ac",
                            16394,
                            "/opt/disk1/postgresql/9.6/data/tablespace/tbs_ac"
                        ],
                        [
                            "tbs_dc",
                            16395,
                            "/opt/disk1/postgresql/9.6/data/tablespace/tbs_dc"
                        ],
                        [
                            "tbs_st",
                            16396,
                            "/opt/disk1/postgresql/9.6/data/tablespace/tbs_st"
                        ]
                    ],
                    "timeline": 1,
                    "version": 90624,
                    "xlog_segment_size": 16777216
                }
            },
            "config": {
                "active": true,
                "archiver": true,
                "archiver_batch_size": 50,
                "backup_directory": "/var/lib/barman/dwh",
                "backup_method": "rsync",
                "backup_options": "concurrent_backup",
                "bandwidth_limit": null,
                "barman_home": "/var/lib/barman",
                "barman_lock_directory": "/var/lib/barman",
                "basebackup_retry_sleep": 30,
                "basebackup_retry_times": 0,
                "basebackups_directory": "/var/lib/barman/dwh/base",
                "check_timeout": 60,
                "compression": null,
                "conninfo": "host=192.168.54.26 user=barman dbname=postgres password=*REDACTED*",
                "create_slot": "manual",
                "custom_compression_filter": null,
                "custom_compression_magic": null,
                "custom_decompression_filter": null,
                "description": "dwh PostgreSQL Database (via SSH)",
                "disabled": false,
                "errors_directory": "/var/lib/barman/dwh/errors",
                "forward_config_path": false,
                "immediate_checkpoint": false,
                "incoming_wals_directory": "/var/lib/barman/dwh/incoming",
                "last_backup_maximum_age": null,
                "last_backup_minimum_size": null,
                "last_wal_maximum_age": null,
                "max_incoming_wals_queue": null,
                "minimum_redundancy": 0,
                "msg_list": [],
                "name": "dwh",
                "network_compression": false,
                "parallel_jobs": 1,
                "path_prefix": "/usr/pgsql-9.6/bin",
                "post_archive_retry_script": null,
                "post_archive_script": null,
                "post_backup_retry_script": null,
                "post_backup_script": "/opt/barman/bin/backup_check.sh",
                "post_delete_retry_script": null,
                "post_delete_script": null,
                "post_recovery_retry_script": null,
                "post_recovery_script": null,
                "post_wal_delete_retry_script": null,
                "post_wal_delete_script": null,
                "pre_archive_retry_script": null,
                "pre_archive_script": null,
                "pre_backup_retry_script": null,
                "pre_backup_script": "/opt/barman/bin/backup_check.sh",
                "pre_delete_retry_script": null,
                "pre_delete_script": null,
                "pre_recovery_retry_script": null,
                "pre_recovery_script": null,
                "pre_wal_delete_retry_script": null,
                "pre_wal_delete_script": null,
                "primary_ssh_command": null,
                "recovery_options": "get-wal",
                "retention_policy": "redundancy 1 b",
                "retention_policy_mode": "auto",
                "reuse_backup": "link",
                "slot_name": null,
                "ssh_command": "ssh post...@192.168.54.26",
                "streaming_archiver": false,
                "streaming_archiver_batch_size": 0,
                "streaming_archiver_name": "barman_receive_wal",
                "streaming_backup_name": "barman_streaming_backup",
                "streaming_conninfo": "host=192.168.54.26 user=barman dbname=postgres password=*REDACTED*",
                "streaming_wals_directory": "/var/lib/barman/dwh/streaming",
                "tablespace_bandwidth_limit": null,
                "wal_retention_policy": "simple-wal 1 b",
                "wals_directory": "/var/lib/barman/dwh/wals"
            },
            "status": {
                "archive_command": "/usr/bin/rsync -a %p barman@barman01:/var/lib/barman/dwh/incoming/%f",
                "archive_mode": "on",
                "archive_timeout": 0,
                "archived_count": 3009993,
                "checkpoint_timeout": 3600,
                "config_file": "/opt/disk1/postgresql/9.6/data/postgresql.conf",
                "current_archived_wals_per_second": 0.23866128463106,
                "current_lsn": "D6C6/9B65D70",
                "current_size": 11065096824178.0,
                "current_xlog": "000000010000D6C600000009",
                "data_checksums": "off",
                "data_directory": "/opt/disk1/postgresql/9.6/data",
                "failed_count": 31171,
                "has_backup_privileges": true,
                "hba_file": "/opt/disk1/postgresql/9.6/data/pg_hba.conf",
                "hot_standby": "off",
                "ident_file": "/opt/disk1/postgresql/9.6/data/pg_ident.conf",
                "included_files": [
                    "/opt/disk1/postgresql/9.6/data/postgresql.auto.conf",
                    "/opt/disk1/postgresql/9.6/data/postgresql-backend_params.conf",
                    "/opt/disk1/postgresql/9.6/data/postgresql-ssl_params.conf"
                ],
                "is_archiving": true,
                "is_in_recovery": false,
                "is_superuser": true,
                "last_archived_time": "Fri Feb 18 13:49:28 2022",
                "last_archived_wal": "000000010000D6C600000008",
                "last_failed_time": "Tue Jan 25 21:22:31 2022",
                "last_failed_wal": "000000010000D0270000001B",
                "max_replication_slots": "2",
                "max_wal_senders": "10",
                "pgespresso_installed": false,
                "postgres_systemid": "6410648610747580763",
                "replication_slot": null,
                "replication_slot_support": true,
                "server_txt_version": "9.6.24",
                "stats_reset": "Wed Sep 22 14:38:30 2021",
                "synchronous_standby_names": [
                    "*"
                ],
                "wal_compression": "off",
                "wal_keep_segments": "5000",
                "wal_level": "replica",
                "xlog_segment_size": 16777216
            },
            "system_info": {
                "kernel_ver": "Linux dwh.XXXX.xxxxx.XXXXX 3.10.0-1160.36.2.el7.x86_64 #1 SMP Wed Jul 21 11:57:15 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux",
                "python_ver": "",
                "release": "RedHat Linux CentOS Linux release 7.9.2009 (Core)",
                "rsync_ver": "rsync  version 3.1.2  protocol version 31",
                "ssh_ver": ""
            },
            "wals": {
                "last_archived_wal_per_timeline": {
                    "00000001": {
                        "compression": null,
                        "name": "000000010000D6C600000008",
                        "size": 16777216,
                        "time": 1645188567.071081
                    }
                }
            }
        },
        "ipropsgaggrdb": {
            "backups": {
                "20220212T173727": {
                    "backup_id": "20220212T173727",
                    "backup_label": "'START WAL LOCATION: 9E4/DC1E8238 (file 00000001000009E4000000DC)\\nCHECKPOINT LOCATION: 9E4/E220B240\\nBACKUP METHOD: streamed\\nBACKUP FROM: master\\nSTART TIME: 2022-02-12 17:22:17 GMT\\nLABEL: Barman backup ipropsgaggrdb 20220212T173727\\n'",
                    "begin_offset": 1999416,
                    "begin_time": "Sat Feb 12 17:37:28 2022",
                    "begin_wal": "00000001000009E4000000DC",
                    "begin_xlog": "9E4/DC1E8238",
                    "config_file": "/srv/disk1/postgresql/9.6/data/postgresql.conf",
                    "copy_stats": {
                        "analysis_time": 6.458109,
                        "analysis_time_per_item": {
                            "aggrtbs": 1.856499,
                            "externaltbs": 0.889107,
                            "historytbs": 1.030563,
                            "indextbs": 0.863507,
                            "pgdata": 1.816359
                        },
                        "copy_time": 3280.974743,
                        "copy_time_per_item": {
                            "aggrtbs": 1737.997175,
                            "externaltbs": 43.976466,
                            "historytbs": 0.953675,
                            "indextbs": 757.529344,
                            "pg_control": 0.496545,
                            "pgdata": 739.993196
                        },
                        "number_of_workers": 1,
                        "serialized_copy_time": 3280.866465,
                        "serialized_copy_time_per_item": {
                            "aggrtbs": 1737.953352,
                            "externaltbs": 43.973859,
                            "historytbs": 0.948529,
                            "indextbs": 757.52442,
                            "pg_control": 0.496545,
                            "pgdata": 739.96976
                        },
                        "total_time": 3287.552677
                    },
                    "deduplicated_size": 334876818756,
                    "end_offset": 7887040,
                    "end_time": "Sat Feb 12 19:17:04 2022",
                    "end_wal": "00000001000009E4000000EC",
                    "end_xlog": "9E4/EC7858C0",
                    "error": null,
                    "hba_file": "/srv/disk1/postgresql/9.6/data/pg_hba.conf",
                    "ident_file": "/srv/disk1/postgresql/9.6/data/pg_ident.conf",
                    "included_files": [
                        "/srv/disk1/postgresql/9.6/data/postgresql.auto.conf",
                        "/srv/disk1/postgresql/9.6/data/postgresql-backend_params.conf",
                        "/srv/disk1/postgresql/9.6/data/postgresql-ssl_params.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "pgdata": "/srv/disk1/postgresql/9.6/data",
                    "server_name": "ipropsgaggrdb",
                    "size": 4388680654590,
                    "status": "DONE",
                    "systemid": "6407417716311790897",
                    "tablespaces": [
                        [
                            "aggrtbs",
                            117419,
                            "/srv/disk1/postgresql/9.6/data/tablespace/aggrtbs"
                        ],
                        [
                            "externaltbs",
                            117420,
                            "/srv/disk1/postgresql/9.6/data/tablespace/externaltbs"
                        ],
                        [
                            "historytbs",
                            117421,
                            "/srv/disk1/postgresql/9.6/data/tablespace/historytbs"
                        ],
                        [
                            "indextbs",
                            117422,
                            "/srv/disk1/postgresql/9.6/data/tablespace/indextbs"
                        ]
                    ],
                    "timeline": 1,
                    "version": 90624,
                    "xlog_segment_size": 16777216
                }
            },
            "config": {
                "active": true,
                "archiver": true,
                "archiver_batch_size": 50,
                "backup_directory": "/var/lib/barman/ipropsgaggrdb",
                "backup_method": "rsync",
                "backup_options": "concurrent_backup",
                "bandwidth_limit": null,
                "barman_home": "/var/lib/barman",
                "barman_lock_directory": "/var/lib/barman",
                "basebackup_retry_sleep": 30,
                "basebackup_retry_times": 0,
                "basebackups_directory": "/var/lib/barman/ipropsgaggrdb/base",
                "check_timeout": 30,
                "compression": null,
                "conninfo": "host=192.168.54.15 user=barman dbname=postgres",
                "create_slot": "manual",
                "custom_compression_filter": null,
                "custom_compression_magic": null,
                "custom_decompression_filter": null,
                "description": "ipropsgaggrdb PostgreSQL Database (via SSH)",
                "disabled": false,
                "errors_directory": "/var/lib/barman/ipropsgaggrdb/errors",
                "forward_config_path": false,
                "immediate_checkpoint": false,
                "incoming_wals_directory": "/var/lib/barman/ipropsgaggrdb/incoming",
                "last_backup_maximum_age": null,
                "last_backup_minimum_size": null,
                "last_wal_maximum_age": null,
                "max_incoming_wals_queue": null,
                "minimum_redundancy": 0,
                "msg_list": [],
                "name": "ipropsgaggrdb",
                "network_compression": false,
                "parallel_jobs": 1,
                "path_prefix": "/usr/pgsql-9.6/bin",
                "post_archive_retry_script": null,
                "post_archive_script": null,
                "post_backup_retry_script": null,
                "post_backup_script": "/opt/barman/bin/backup_check.sh",
                "post_delete_retry_script": null,
                "post_delete_script": null,
                "post_recovery_retry_script": null,
                "post_recovery_script": null,
                "post_wal_delete_retry_script": null,
                "post_wal_delete_script": null,
                "pre_archive_retry_script": null,
                "pre_archive_script": null,
                "pre_backup_retry_script": null,
                "pre_backup_script": "/opt/barman/bin/backup_check.sh",
                "pre_delete_retry_script": null,
                "pre_delete_script": null,
                "pre_recovery_retry_script": null,
                "pre_recovery_script": null,
                "pre_wal_delete_retry_script": null,
                "pre_wal_delete_script": null,
                "primary_ssh_command": null,
                "recovery_options": "get-wal",
                "retention_policy": "redundancy 1 b",
                "retention_policy_mode": "auto",
                "reuse_backup": "link",
                "slot_name": null,
                "ssh_command": "ssh post...@192.168.54.15",
                "streaming_archiver": false,
                "streaming_archiver_batch_size": 0,
                "streaming_archiver_name": "barman_receive_wal",
                "streaming_backup_name": "barman_streaming_backup",
                "streaming_conninfo": "host=192.168.54.15 user=barman dbname=postgres",
                "streaming_wals_directory": "/var/lib/barman/ipropsgaggrdb/streaming",
                "tablespace_bandwidth_limit": null,
                "wal_retention_policy": "simple-wal 1 b",
                "wals_directory": "/var/lib/barman/ipropsgaggrdb/wals"
            },
            "status": {
                "archive_command": "rsync -a %p bar...@192.168.54.17:/var/lib/barman/ipropsgaggrdb/incoming/%f",
                "archive_mode": "on",
                "archive_timeout": 0,
                "archived_count": 215812,
                "checkpoint_timeout": 3600,
                "config_file": "/srv/disk1/postgresql/9.6/data/postgresql.conf",
                "current_archived_wals_per_second": 0.00658988457511581,
                "current_lsn": "9EC/20248000",
                "current_size": 4401695673208.0,
                "current_xlog": "00000001000009EC00000020",
                "data_checksums": "off",
                "data_directory": "/srv/disk1/postgresql/9.6/data",
                "failed_count": 12318,
                "has_backup_privileges": true,
                "hba_file": "/srv/disk1/postgresql/9.6/data/pg_hba.conf",
                "hot_standby": "on",
                "ident_file": "/srv/disk1/postgresql/9.6/data/pg_ident.conf",
                "included_files": [
                    "/srv/disk1/postgresql/9.6/data/postgresql.auto.conf",
                    "/srv/disk1/postgresql/9.6/data/postgresql-backend_params.conf",
                    "/srv/disk1/postgresql/9.6/data/postgresql-ssl_params.conf"
                ],
                "is_archiving": true,
                "is_in_recovery": false,
                "is_superuser": true,
                "last_archived_time": "Fri Feb 18 13:57:54 2022",
                "last_archived_wal": "00000001000009EC0000001F",
                "last_failed_time": "Tue Jan 25 21:18:09 2022",
                "last_failed_wal": "00000001000009CC00000018",
                "max_replication_slots": "2",
                "max_wal_senders": "10",
                "pgespresso_installed": false,
                "postgres_systemid": "6407417716311790897",
                "replication_slot": null,
                "replication_slot_support": true,
                "server_txt_version": "9.6.24",
                "stats_reset": "Thu Feb  4 19:01:59 2021",
                "synchronous_standby_names": [
                    ""
                ],
                "wal_compression": "off",
                "wal_keep_segments": "5000",
                "wal_level": "replica",
                "xlog_segment_size": 16777216
            },
            "system_info": {
                "kernel_ver": "Linux ipropsgaggr01.XXXX.xxxxx.XXXXX 3.10.0-1127.10.1.el7.x86_64 #1 SMP Wed Jun 3 14:28:03 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux",
                "python_ver": "",
                "release": "LSB Version:\t:core-4.1-amd64:core-4.1-noarch\nDistributor ID:\tCentOS\nDescription:\tCentOS Linux release 7.9.2009 (Core)\nRelease:\t7.9.2009\nCodename:\tCore",
                "rsync_ver": "rsync  version 3.1.2  protocol version 31",
                "ssh_ver": ""
            },
            "wals": {
                "last_archived_wal_per_timeline": {
                    "00000001": {
                        "compression": null,
                        "name": "00000001000009EC0000001F",
                        "size": 16777216,
                        "time": 1645189073.443019
                    }
                }
            }
        }
    }
}

thanks thanks thanks







--

Jose Manuel Valseca                            

 Asset & Recovery

 EUIGS - Admiral Group

#LeadingTheFuture #BusinessLover

Awesome Job? 

--
--
You received this message because you are subscribed to the "Barman for PostgreSQL" group.
To post to this group, send email to pgba...@googlegroups.com
To unsubscribe from this group, send email to
pgbarman+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/pgbarman?hl=en?hl=en-GB

---
You received this message because you are subscribed to the Google Groups "Barman, Backup and Recovery Manager for PostgreSQL" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pgbarman+u...@googlegroups.com.
Message has been deleted

Michael Wallace

unread,
Feb 21, 2022, 4:45:16 AM2/21/22
to pgba...@googlegroups.com
Thanks Jose - I just wanted to eliminate that as a possible factor.

I haven't been able to reproduce the issue locally running CentOS 7 and PostgreSQL 9.6. I'm also not convinced the symlinked pg_xlog is a factor either - when you recover the backup the pg_xlog will be a directory and not a symlink so you'd need to manually recreate that, but it shouldn't affect the restore of the WALs.

Could you try enabling debug logging and restarting postgres from a restore, then posting the output? Adding `log_min_messages = debug5` to postgresql.conf should do the trick.

On Sat, Feb 19, 2022 at 9:04 AM Jose Manuel Valseca Echevarria <josemanue...@euigs.com> wrote:
Sorry about that. Those blanks must have come from modifying the file. Because of confidentiality I had to modify it. If you are so kind as to provide me with an e-mail address, I can provide you with the original content.

Best regards



--

Jose Manuel Valseca                            

 Asset & Recovery

 EUIGS - Admiral Group

#LeadingTheFuture #BusinessLover

Awesome Job? 

--
--
You received this message because you are subscribed to the "Barman for PostgreSQL" group.
To post to this group, send email to pgba...@googlegroups.com
To unsubscribe from this group, send email to
pgbarman+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/pgbarman?hl=en?hl=en-GB

---
You received this message because you are subscribed to the Google Groups "Barman, Backup and Recovery Manager for PostgreSQL" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pgbarman+u...@googlegroups.com.
Message has been deleted
Message has been deleted

Luca Ferrari

unread,
Feb 21, 2022, 11:05:46 AM2/21/22
to Barman, Backup and Recovery Manager for PostgreSQL
On Mon, Feb 21, 2022 at 4:56 PM Jose Manuel Valseca Echevarria
<josemanue...@euigs.com> wrote:
>
> More information. I attach a screenshot of the pg_xlog directory on the production server. In the barman backup directory this directory is empty.
>

Please, don't post screenshots! Copy and paste the text, it will ease
the work for who is trying to help you and for future searching for on
the mailing list.
Having said that, I see that PostgreSQL is trying to replay a
0x10x14670xE8 WAL file while in your pg_xlog directory you have
0x10x14700x0B and more recent files.
Seems to me the required files are not transferred or the recovery
command is wrong somehow, but I cannot figure out what is going wrong.

Luca
Message has been deleted

Luca Ferrari

unread,
Feb 21, 2022, 11:23:50 AM2/21/22
to Barman, Backup and Recovery Manager for PostgreSQL
On Mon, Feb 21, 2022 at 5:14 PM Jose Manuel Valseca Echevarria
<josemanue...@euigs.com> wrote:
>
> Sorry. I got this information from a console from which I can't copy and paste.
> What do you think are the steps I should follow?

Unluckily I've no idea. I read in a previous post that you have a link
on pg_xlog, any chance it is causing some sort of problem? Can you try
without the link?
I see also from the diagnose output that your last archived wal is far
behind what the system is expecting: "last_archived_wal":
"00000001000014650000007F",. I suspect have screenshotted a different
restore than the diagnose output, therefore could you check what is
the last_archived_wal of the restore you are trying to restore? Also
the first wal would help to see what barman thinks there's in the
backup.

>
> Thank you very much you are a great team

Just to be clear, I'm not a part of barman team. It just happens I do
reply to the mailing list.

Luca

Michael Wallace

unread,
Feb 21, 2022, 1:48:58 PM2/21/22
to pgba...@googlegroups.com
When I try to reproduce this locally I can see logs of the restore_command executing which are not present in the logs you posted. Taking a look at the PostgreSQL source shows that the only code path where the message "could not locate a valid checkpoint record" occurs is in the code path which executes if the backup label cannot be read [1].

As an experiment I deleted backup_label before recovering and was able to reproduce the logs you are seeing very closely:

    LOG:  database system was interrupted; last known up at 2022-02-21 19:23:03 CET
    DEBUG:  restore_command = 'barman-wal-restore -P -U barman backup main %f %p'
    LOG:  starting archive recovery
    DEBUG:  could not open file "pg_xlog/00000001000000000000000A": No such file or directory

    LOG:  invalid primary checkpoint record
    DEBUG:  could not open file "pg_xlog/000000010000000000000009": No such file or directory

    LOG:  invalid secondary checkpoint record
    PANIC:  could not locate a valid checkpoint record

Which raises the following questions:

- Is there a reason why PostgreSQL wouldn't be able to find the backup_label in your recovered PGDATA directory?
- When `barman recover` has completed, is there a backup_label present in the recovery target directory? If so, what does it contain?
- If you restart PostgreSQL with the backup_label in place, what happens?


--
--
You received this message because you are subscribed to the "Barman for PostgreSQL" group.
To post to this group, send email to pgba...@googlegroups.com
To unsubscribe from this group, send email to
pgbarman+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/pgbarman?hl=en?hl=en-GB

---
You received this message because you are subscribed to the Google Groups "Barman, Backup and Recovery Manager for PostgreSQL" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pgbarman+u...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages