ERROR: Impossible to start the backup. Check the log for more details, or run 'barman check CDUP01'
1,695 views
Skip to first unread message
Jan Birk
Mar 25, 2016, 7:45:37 AM3/25/16
to Barman, Backup and Recovery Manager for PostgreSQL
Hi,
Where can I look up some more info the error in subj? The backup is failing for all 4 servers in /etc/backup.d. (All 4 servers are test servers). 3 things has happened since the last successful test backup:
1) I ran out space on the barman server (reduced retention and ran delete server oldest with succes)
2) I ran out of space on one pg server and ran pg_resetxlog
3) Upgraded from 1.5.x to 1.6.0
Now I have this:
$ barman backup CDUP01
is resulting in
ERROR: Impossible to start the backup. Check the log for more details, or run 'barman check CDUP01'. The logfile just tell the same. A check gives:
$ barman check CDUP01
Server CDUP01:
PostgreSQL: OK
directories: OK
retention policy settings: OK
backup maximum age: OK (no last_backup_maximum_age provided)
compression settings: OK
failed backups: FAILED (there are 1 failed backups)
minimum redundancy requirements: OK (have 0 backups, expected at least 0)
ssh: OK (PostgreSQL server)
not in recovery: OK
archive_mode: OK
archive_command: OK
archiver errors: OK
$ barman show-server CDUP01
Server CDUP01:
active: True
archive_command: rsync -a %p barman@psql01....:/data/barman/CDUP01/incoming
archive_mode: on
archiver: True
backup_directory: /data/barman/CDUP01
backup_method: rsync
backup_options: BackupOptions(['exclusive_backup'])
bandwidth_limit: None
basebackup_retry_sleep: 30
basebackup_retry_times: 0
basebackups_directory: /data/barman/CDUP01/base
compression: gzip
config_file: /var/lib/pgsql/data/postgresql.conf
conninfo: host=cdup........ user=postgres
current_size: 23828027
current_xlog: 0000000100000006000000B4
custom_compression_filter: None
custom_decompression_filter: None
data_directory: /var/lib/pgsql/data
description: CDUP01 TEST
disabled: False
errors_directory: /data/barman/CDUP01/errors
hba_file: /var/lib/pgsql/data/pg_hba.conf
ident_file: /var/lib/pgsql/data/pg_ident.conf
immediate_checkpoint: False
incoming_wals_directory: /data/barman/CDUP01/incoming
last_archived_wal: None
last_backup_maximum_age: None
minimum_redundancy: 0
network_compression: False
path_prefix: None
pgespresso_installed: False
post_archive_retry_script: None
post_archive_script: None
post_backup_retry_script: None
post_backup_script: None
pre_archive_retry_script: None
pre_archive_script: None
pre_backup_retry_script: None
pre_backup_script: None
recovery_options: RecoveryOptions([])
retention_policy: RECOVERY WINDOW OF 2 DAYS
retention_policy_mode: auto
reuse_backup: None
server_txt_version: 8.4.20
ssh_command: ssh postgres@cdup01......
streaming_archiver: False
streaming_conninfo: host=cdup01........ user=postgres
streaming_wals_directory: /data/barman/CDUP01/streaming
tablespace_bandwidth_limit: None
wal_retention_policy: MAIN
wals_directory: /data/barman/CDUP01/wals
-------------------------------
Thx and best regards
/Jan
Where can I look up some more info the error in subj? The backup is failing for all 4 servers in /etc/backup.d. (All 4 servers are test servers). 3 things has happened since the last successful test backup:
1) I ran out space on the barman server (reduced retention and ran delete server oldest with succes)
2) I ran out of space on one pg server and ran pg_resetxlog
3) Upgraded from 1.5.x to 1.6.0
Now I have this:
$ barman backup CDUP01
is resulting in
ERROR: Impossible to start the backup. Check the log for more details, or run 'barman check CDUP01'. The logfile just tell the same. A check gives:
$ barman check CDUP01
Server CDUP01:
PostgreSQL: OK
directories: OK
retention policy settings: OK
backup maximum age: OK (no last_backup_maximum_age provided)
compression settings: OK
failed backups: FAILED (there are 1 failed backups)
minimum redundancy requirements: OK (have 0 backups, expected at least 0)
ssh: OK (PostgreSQL server)
not in recovery: OK
archive_mode: OK
archive_command: OK
archiver errors: OK
$ barman show-server CDUP01
Server CDUP01:
active: True
archive_command: rsync -a %p barman@psql01....:/data/barman/CDUP01/incoming
archive_mode: on
archiver: True
backup_directory: /data/barman/CDUP01
backup_method: rsync
backup_options: BackupOptions(['exclusive_backup'])
bandwidth_limit: None
basebackup_retry_sleep: 30
basebackup_retry_times: 0
basebackups_directory: /data/barman/CDUP01/base
compression: gzip
config_file: /var/lib/pgsql/data/postgresql.conf
conninfo: host=cdup........ user=postgres
current_size: 23828027
current_xlog: 0000000100000006000000B4
custom_compression_filter: None
custom_decompression_filter: None
data_directory: /var/lib/pgsql/data
description: CDUP01 TEST
disabled: False
errors_directory: /data/barman/CDUP01/errors
hba_file: /var/lib/pgsql/data/pg_hba.conf
ident_file: /var/lib/pgsql/data/pg_ident.conf
immediate_checkpoint: False
incoming_wals_directory: /data/barman/CDUP01/incoming
last_archived_wal: None
last_backup_maximum_age: None
minimum_redundancy: 0
network_compression: False
path_prefix: None
pgespresso_installed: False
post_archive_retry_script: None
post_archive_script: None
post_backup_retry_script: None
post_backup_script: None
pre_archive_retry_script: None
pre_archive_script: None
pre_backup_retry_script: None
pre_backup_script: None
recovery_options: RecoveryOptions([])
retention_policy: RECOVERY WINDOW OF 2 DAYS
retention_policy_mode: auto
reuse_backup: None
server_txt_version: 8.4.20
ssh_command: ssh postgres@cdup01......
streaming_archiver: False
streaming_conninfo: host=cdup01........ user=postgres
streaming_wals_directory: /data/barman/CDUP01/streaming
tablespace_bandwidth_limit: None
wal_retention_policy: MAIN
wals_directory: /data/barman/CDUP01/wals
-------------------------------
Thx and best regards
/Jan
Giulio Calacoci
Mar 25, 2016, 12:57:59 PM3/25/16
to pgbarman
Hi Jan,
As far as I can see, the backup is failing to start because of this
failed check:
failed backups: FAILED (there are 1 failed backups)
to pass this check the backup catalog of your server must not contain
FAILED backups.
/We have fixed this behavior just after te 1.6.0 release and the patch
is on the master branch on github,
you can find it here:
https://github.com/2ndquadrant-it/barman/commit/72ba9e4d0afe366d68954eb84ae8381cfdf58c5e
besides this failure everything else seems Ok.
Could you please provide the output of the 'barman diagnose' command?
An important note:
after a pg_resetxlog the postgresql data directory should be
considered compromised.
as you can see here:
http://www.postgresql.org/docs/9.5/static/app-pgresetxlog.html
the pg_resexlog documentation states that:
"After running this command, it should be possible to start the server,
but bear in mind that the database might contain inconsistent data due
to partially-committed transactions.
You should immediately dump your data, run initdb, and reload.
After reload, check for inconsistencies and repair as needed."
I strongly advise against using pg_resexlog unless is your last resort
and you know VERY DEEPLY what you are doing
Ciao,
Giulio
> --
> --
> You received this message because you are subscribed to the "Barman for
> PostgreSQL" group.
> To post to this group, send email to pgba...@googlegroups.com
> To unsubscribe from this group, send email to
> pgbarman+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/pgbarman?hl=en?hl=en-GB
>
> ---
> You received this message because you are subscribed to the Google Groups
> "Barman, Backup and Recovery Manager for PostgreSQL" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pgbarman+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
--
Giulio Calacoci - 2ndQuadrant Italia
PostgreSQL Training, Services and Support
giulio....@2ndQuadrant.it | www.2ndQuadrant.it
As far as I can see, the backup is failing to start because of this
failed check:
failed backups: FAILED (there are 1 failed backups)
FAILED backups.
/We have fixed this behavior just after te 1.6.0 release and the patch
is on the master branch on github,
you can find it here:
https://github.com/2ndquadrant-it/barman/commit/72ba9e4d0afe366d68954eb84ae8381cfdf58c5e
besides this failure everything else seems Ok.
Could you please provide the output of the 'barman diagnose' command?
An important note:
after a pg_resetxlog the postgresql data directory should be
considered compromised.
as you can see here:
http://www.postgresql.org/docs/9.5/static/app-pgresetxlog.html
the pg_resexlog documentation states that:
"After running this command, it should be possible to start the server,
but bear in mind that the database might contain inconsistent data due
to partially-committed transactions.
You should immediately dump your data, run initdb, and reload.
After reload, check for inconsistencies and repair as needed."
I strongly advise against using pg_resexlog unless is your last resort
and you know VERY DEEPLY what you are doing
Ciao,
Giulio
> --
> You received this message because you are subscribed to the "Barman for
> PostgreSQL" group.
> To post to this group, send email to pgba...@googlegroups.com
> To unsubscribe from this group, send email to
> pgbarman+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/pgbarman?hl=en?hl=en-GB
>
> ---
> You received this message because you are subscribed to the Google Groups
> "Barman, Backup and Recovery Manager for PostgreSQL" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pgbarman+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
--
Giulio Calacoci - 2ndQuadrant Italia
PostgreSQL Training, Services and Support
giulio....@2ndQuadrant.it | www.2ndQuadrant.it
Jan Birk
Mar 26, 2016, 3:28:33 AM3/26/16
to Barman, Backup and Recovery Manager for PostgreSQL
Hi Giulio,
Thanks for your answers. Output from diagnose at the end. I have patched server.py and it its working fine again :-) Thanks!
And thanks for your comment about pg_resetxlog. I will do as the fine manual describes :-)
Best
/Jan
------------------------ diagnose ---------------
-bash-4.1$ barman diagnose
{
"global": {
"config": {
"barman_home": "/data/barman",
"barman_user": "barman",
"configuration_files_directory": "/etc/barman.d",
"errors_list": [],
"log_file": "/var/log/barman/barman.log"
},
"system_info": {
"barman_ver": "1.6.0",
"kernel_ver": "Linux psql01.stil.dk 2.6.39-400.250.5.el6uek.x86_64 #1 SMP Tue Jun 9 16:48:23 PDT 2015 x86_64 x86_64 x86_64 GNU/Linux",
"python_ver": "Python 2.6.6",
"release": "RedHat Linux Red Hat Enterprise Linux Server release 6.7 (Santiago)",
"rsync_ver": "rsync version 3.0.6 protocol version 30",
"ssh_ver": "OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013"
}
},
"servers": {
"CDUP01": {
"backups": {
"20160107T075815": {
"backup_id": "20160107T075815",
"backup_label": null,
"begin_offset": 32,
"begin_time": "Thu Jan 7 07:54:29 2016",
"begin_wal": "00000001000000000000001F",
"begin_xlog": "0/1F000020",
"config_file": "/var/lib/pgsql/data/postgresql.conf",
"deduplicated_size": null,
"end_offset": null,
"end_time": null,
"end_wal": null,
"end_xlog": null,
"error": null,
"hba_file": "/var/lib/pgsql/data/pg_hba.conf",
"ident_file": "/var/lib/pgsql/data/pg_ident.conf",
"included_files": null,
"mode": "default",
"pgdata": "/var/lib/pgsql/data",
"server_name": "CDUP01",
"size": null,
"status": "STARTED",
"tablespaces": null,
"timeline": 1,
"version": 80420
},
"20160315T113936": {
"backup_id": "20160315T113936",
"backup_label": null,
"begin_offset": 32,
"begin_time": "Tue Mar 15 11:34:57 2016",
"begin_wal": "000000010000000600000099",
"begin_xlog": "6/99000020",
"config_file": "/var/lib/pgsql/data/postgresql.conf",
"deduplicated_size": null,
"end_offset": null,
"end_time": null,
"end_wal": null,
"end_xlog": null,
"error": null,
"hba_file": "/var/lib/pgsql/data/pg_hba.conf",
"ident_file": "/var/lib/pgsql/data/pg_ident.conf",
"included_files": null,
"mode": "default",
"pgdata": "/var/lib/pgsql/data",
"server_name": "CDUP01",
"size": null,
"status": "STARTED",
"tablespaces": null,
"timeline": 1,
"version": 80420
},
"20160315T133355": {
"backup_id": "20160315T133355",
"backup_label": null,
"begin_offset": 32,
"begin_time": "Tue Mar 15 13:29:16 2016",
"begin_wal": "00000001000000060000009C",
"begin_xlog": "6/9C000020",
"config_file": "/var/lib/pgsql/data/postgresql.conf",
"deduplicated_size": null,
"end_offset": null,
"end_time": null,
"end_wal": null,
"end_xlog": null,
"error": "failure issuing start backup command (Cannot terminate exclusive backup. You might have to manually execute pg_stop_backup() on your PostgreSQL server)",
"hba_file": "/var/lib/pgsql/data/pg_hba.conf",
"ident_file": "/var/lib/pgsql/data/pg_ident.conf",
"included_files": null,
"mode": "default",
"pgdata": "/var/lib/pgsql/data",
"server_name": "CDUP01",
"size": null,
"status": "FAILED",
"tablespaces": null,
"timeline": 1,
"version": 80420
}
},
"config": {
"active": true,
"archiver": true,
"backup_directory": "/data/barman/CDUP01",
"backup_method": "rsync",
"backup_options": "exclusive_backup",
"bandwidth_limit": null,
"barman_home": "/data/barman",
"barman_lock_directory": "/data/barman",
"basebackup_retry_sleep": 30,
"basebackup_retry_times": 0,
"basebackups_directory": "/data/barman/CDUP01/base",
"compression": "gzip",
"conninfo": "host=cdup01.easyc.uni-c.dk user=postgres",
"custom_compression_filter": null,
"custom_decompression_filter": null,
"description": "CDUP01 TEST",
"disabled": false,
"errors_directory": "/data/barman/CDUP01/errors",
"immediate_checkpoint": false,
"incoming_wals_directory": "/data/barman/CDUP01/incoming",
"last_backup_maximum_age": null,
"minimum_redundancy": 0,
"msg_list": [],
"name": "CDUP01",
"network_compression": false,
"path_prefix": null,
"post_archive_retry_script": null,
"post_archive_script": null,
"post_backup_retry_script": null,
"post_backup_script": null,
"pre_archive_retry_script": null,
"pre_archive_script": null,
"pre_backup_retry_script": null,
"pre_backup_script": null,
"recovery_options": "",
"retention_policy": "window 2 d",
"retention_policy_mode": "auto",
"reuse_backup": null,
"ssh_command": "ssh post...@cdup01.easyc.uni-c.dk",
"streaming_archiver": false,
"streaming_conninfo": "host=cdup01.easyc.uni-c.dk user=postgres",
"streaming_wals_directory": "/data/barman/CDUP01/streaming",
"tablespace_bandwidth_limit": null,
"wal_retention_policy": "simple-wal 2 d",
"wals_directory": "/data/barman/CDUP01/wals"
},
"status": {
"archive_command": "rsync -a %p bar...@psql01.stil.dk:/data/barman/CDUP01/incoming",
"archive_mode": "on",
"config_file": "/var/lib/pgsql/data/postgresql.conf",
"current_size": 23836219.0,
"current_xlog": "0000000100000006000000B4",
"data_directory": "/var/lib/pgsql/data",
"hba_file": "/var/lib/pgsql/data/pg_hba.conf",
"ident_file": "/var/lib/pgsql/data/pg_ident.conf",
"last_archived_wal": null,
"pgespresso_installed": false,
"server_txt_version": "8.4.20"
},
"system_info": {
"kernel_ver": "Linux cdup01.easyc.uni-c.dk 2.6.39-400.215.2.el6uek.x86_64 #1 SMP Fri Jun 6 12:51:44 PDT 2014 x86_64 x86_64 x86_64 GNU/Linux",
"python_ver": "Python 2.6.6",
"release": "RedHat Linux Red Hat Enterprise Linux Server release 6.5 (Santiago)",
"rsync_ver": "rsync version 3.0.6 protocol version 30",
"ssh_ver": "OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013"
}
},
"CDUP02": {
"backups": {
"20151228T134927": {
"backup_id": "20151228T134927",
"backup_label": null,
"begin_offset": null,
"begin_time": null,
"begin_wal": null,
"begin_xlog": null,
"config_file": "/var/lib/pgsql/data/postgresql.conf",
"deduplicated_size": null,
"end_offset": null,
"end_time": null,
"end_wal": null,
"end_xlog": null,
"error": "failure issuing start backup command (pg_start_backup(): a backup is already in progress)",
"hba_file": "/var/lib/pgsql/data/pg_hba.conf",
"ident_file": "/var/lib/pgsql/data/pg_ident.conf",
"included_files": null,
"mode": "default",
"pgdata": "/var/lib/pgsql/data",
"server_name": "CDUP02",
"size": null,
"status": "FAILED",
"tablespaces": null,
"timeline": null,
"version": 80420
},
"20160325T104632": {
"backup_id": "20160325T104632",
"backup_label": null,
"begin_offset": 32,
"begin_time": "Fri Mar 25 10:46:32 2016",
"begin_wal": "000000010000000100000043",
"begin_xlog": "1/43000020",
"config_file": "/var/lib/pgsql/data/postgresql.conf",
"deduplicated_size": null,
"end_offset": null,
"end_time": null,
"end_wal": null,
"end_xlog": null,
"error": null,
"hba_file": "/var/lib/pgsql/data/pg_hba.conf",
"ident_file": "/var/lib/pgsql/data/pg_ident.conf",
"included_files": null,
"mode": "default",
"pgdata": "/var/lib/pgsql/data",
"server_name": "CDUP02",
"size": null,
"status": "STARTED",
"tablespaces": null,
"timeline": 1,
"version": 80420
}
},
"config": {
"active": true,
"archiver": true,
"backup_directory": "/data/barman/CDUP02",
"backup_method": "rsync",
"backup_options": "exclusive_backup",
"bandwidth_limit": null,
"barman_home": "/data/barman",
"barman_lock_directory": "/data/barman",
"basebackup_retry_sleep": 30,
"basebackup_retry_times": 0,
"basebackups_directory": "/data/barman/CDUP02/base",
"compression": "gzip",
"conninfo": "host=cdup02.easyc.uni-c.dk user=postgres",
"custom_compression_filter": null,
"custom_decompression_filter": null,
"description": "CDUP02 PROD",
"disabled": false,
"errors_directory": "/data/barman/CDUP02/errors",
"immediate_checkpoint": false,
"incoming_wals_directory": "/data/barman/CDUP02/incoming",
"last_backup_maximum_age": null,
"minimum_redundancy": 0,
"msg_list": [],
"name": "CDUP02",
"network_compression": false,
"path_prefix": null,
"post_archive_retry_script": null,
"post_archive_script": null,
"post_backup_retry_script": null,
"post_backup_script": null,
"pre_archive_retry_script": null,
"pre_archive_script": null,
"pre_backup_retry_script": null,
"pre_backup_script": null,
"recovery_options": "",
"retention_policy": "window 1 w",
"retention_policy_mode": "auto",
"reuse_backup": null,
"ssh_command": "ssh post...@cdup02.easyc.uni-c.dk",
"streaming_archiver": false,
"streaming_conninfo": "host=cdup02.easyc.uni-c.dk user=postgres",
"streaming_wals_directory": "/data/barman/CDUP02/streaming",
"tablespace_bandwidth_limit": null,
"wal_retention_policy": "simple-wal 1 w",
"wals_directory": "/data/barman/CDUP02/wals"
},
"status": {
"archive_command": "rsync -a %p bar...@psql01.stil.dk:/data/barman/CDUP02/incoming",
"archive_mode": "on",
"config_file": "/var/lib/pgsql/data/postgresql.conf",
"current_size": 36881883.0,
"current_xlog": "000000010000000100000044",
"data_directory": "/var/lib/pgsql/data",
"hba_file": "/var/lib/pgsql/data/pg_hba.conf",
"ident_file": "/var/lib/pgsql/data/pg_ident.conf",
"last_archived_wal": "000000010000000100000043.00000020.backup",
"pgespresso_installed": false,
"server_txt_version": "8.4.20"
},
"system_info": {
"kernel_ver": "Linux cdup02.easyc.uni-c.dk 2.6.39-400.215.2.el6uek.x86_64 #1 SMP Fri Jun 6 12:51:44 PDT 2014 x86_64 x86_64 x86_64 GNU/Linux",
"python_ver": "Python 2.6.6",
"release": "RedHat Linux Red Hat Enterprise Linux Server release 6.5 (Santiago)",
"rsync_ver": "rsync version 3.0.6 protocol version 30",
"ssh_ver": "OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013"
}
},
"CDUP03": {
"backups": {
"20160318T125416": {
"backup_id": "20160318T125416",
"backup_label": null,
"begin_offset": null,
"begin_time": null,
"begin_wal": null,
"begin_xlog": null,
"config_file": "/var/lib/pgsql/data/postgresql.conf",
"deduplicated_size": null,
"end_offset": null,
"end_time": null,
"end_wal": null,
"end_xlog": null,
"error": "failure issuing start backup command (pg_start_backup(): server closed the connection unexpectedly)",
"hba_file": "/var/lib/pgsql/data/pg_hba.conf",
"ident_file": "/var/lib/pgsql/data/pg_ident.conf",
"included_files": null,
"mode": "default",
"pgdata": "/var/lib/pgsql/data",
"server_name": "CDUP03",
"size": null,
"status": "FAILED",
"tablespaces": null,
"timeline": null,
"version": 80420
}
},
"config": {
"active": true,
"archiver": true,
"backup_directory": "/data/barman/CDUP03",
"backup_method": "rsync",
"backup_options": "exclusive_backup",
"bandwidth_limit": null,
"barman_home": "/data/barman",
"barman_lock_directory": "/data/barman",
"basebackup_retry_sleep": 30,
"basebackup_retry_times": 0,
"basebackups_directory": "/data/barman/CDUP03/base",
"compression": "gzip",
"conninfo": "host=195.231.251.156 user=postgres",
"custom_compression_filter": null,
"custom_decompression_filter": null,
"description": "CDUP PRE-PROD",
"disabled": false,
"errors_directory": "/data/barman/CDUP03/errors",
"immediate_checkpoint": false,
"incoming_wals_directory": "/data/barman/CDUP03/incoming",
"last_backup_maximum_age": null,
"minimum_redundancy": 0,
"msg_list": [],
"name": "CDUP03",
"network_compression": false,
"path_prefix": null,
"post_archive_retry_script": null,
"post_archive_script": null,
"post_backup_retry_script": null,
"post_backup_script": null,
"pre_archive_retry_script": null,
"pre_archive_script": null,
"pre_backup_retry_script": null,
"pre_backup_script": null,
"recovery_options": "",
"retention_policy": "window 1 w",
"retention_policy_mode": "auto",
"reuse_backup": null,
"ssh_command": "ssh post...@195.231.251.156",
"streaming_archiver": false,
"streaming_conninfo": "host=195.231.251.156 user=postgres",
"streaming_wals_directory": "/data/barman/CDUP03/streaming",
"tablespace_bandwidth_limit": null,
"wal_retention_policy": "simple-wal 1 w",
"wals_directory": "/data/barman/CDUP03/wals"
},
"status": {
"archive_command": "rsync -a %p bar...@psql01.stil.dk:/data/barman/CDUP03/incoming",
"archive_mode": "on",
"config_file": "/var/lib/pgsql/data/postgresql.conf",
"current_size": 27670467.0,
"current_xlog": "00000001000000070000009E",
"data_directory": "/var/lib/pgsql/data",
"hba_file": "/var/lib/pgsql/data/pg_hba.conf",
"ident_file": "/var/lib/pgsql/data/pg_ident.conf",
"last_archived_wal": "00000001000000070000009D",
"pgespresso_installed": false,
"server_txt_version": "8.4.20"
},
"system_info": {
"kernel_ver": "Linux cdup03.easyc.uni-c.dk 2.6.39-400.215.2.el6uek.x86_64 #1 SMP Fri Jun 6 12:51:44 PDT 2014 x86_64 x86_64 x86_64 GNU/Linux",
"python_ver": "Python 2.6.6",
"release": "RedHat Linux Red Hat Enterprise Linux Server release 6.5 (Santiago)",
"rsync_ver": "rsync version 3.0.6 protocol version 30",
"ssh_ver": "OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013"
}
},
"OrdblindetestTEST": {
"backups": {},
"config": {
"active": true,
"archiver": true,
"backup_directory": "/data/barman/OrdblindetestTEST",
"backup_method": "rsync",
"backup_options": "exclusive_backup",
"bandwidth_limit": null,
"barman_home": "/data/barman",
"barman_lock_directory": "/data/barman",
"basebackup_retry_sleep": 30,
"basebackup_retry_times": 0,
"basebackups_directory": "/data/barman/OrdblindetestTEST/base",
"compression": "gzip",
"conninfo": "host=test.ordblindetest.nu user=postgres",
"custom_compression_filter": null,
"custom_decompression_filter": null,
"description": "Ordblindetest TEST",
"disabled": false,
"errors_directory": "/data/barman/OrdblindetestTEST/errors",
"immediate_checkpoint": false,
"incoming_wals_directory": "/data/barman/OrdblindetestTEST/incoming",
"last_backup_maximum_age": null,
"minimum_redundancy": 0,
"msg_list": [],
"name": "OrdblindetestTEST",
"network_compression": false,
"path_prefix": null,
"post_archive_retry_script": null,
"post_archive_script": null,
"post_backup_retry_script": null,
"post_backup_script": null,
"pre_archive_retry_script": null,
"pre_archive_script": null,
"pre_backup_retry_script": null,
"pre_backup_script": null,
"recovery_options": "",
"retention_policy": "window 1 w",
"retention_policy_mode": "auto",
"reuse_backup": null,
"ssh_command": "ssh post...@test.ordblindetest.nu",
"streaming_archiver": false,
"streaming_conninfo": "host=test.ordblindetest.nu user=postgres",
"streaming_wals_directory": "/data/barman/OrdblindetestTEST/streaming",
"tablespace_bandwidth_limit": null,
"wal_retention_policy": "simple-wal 1 w",
"wals_directory": "/data/barman/OrdblindetestTEST/wals"
},
"status": {
"archive_command": null,
"archive_mode": null,
"current_xlog": null,
"data_directory": null,
"pgespresso_installed": null,
"server_txt_version": null
},
"system_info": {
"kernel_ver": "Linux test01.ordblindetest.nu 3.8.13-26.2.4.el6uek.x86_64 #2 SMP Fri May 9 11:36:29 PDT 2014 x86_64 x86_64 x86_64 GNU/Linux",
"python_ver": "Python 2.6.6",
"release": "LSB Version:\t:base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch\nDistributor ID:\tOracleServer\nDescription:\tOracle Linux Server release 6.5\nRelease:\t6.5\nCodename:\tn/a",
"rsync_ver": "rsync version 3.0.6 protocol version 30",
"ssh_ver": "OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013"
}
}
}
}
---------------------------------------------------
Thanks for your answers. Output from diagnose at the end. I have patched server.py and it its working fine again :-) Thanks!
And thanks for your comment about pg_resetxlog. I will do as the fine manual describes :-)
Best
/Jan
------------------------ diagnose ---------------
-bash-4.1$ barman diagnose
{
"global": {
"config": {
"barman_home": "/data/barman",
"barman_user": "barman",
"configuration_files_directory": "/etc/barman.d",
"errors_list": [],
"log_file": "/var/log/barman/barman.log"
},
"system_info": {
"barman_ver": "1.6.0",
"kernel_ver": "Linux psql01.stil.dk 2.6.39-400.250.5.el6uek.x86_64 #1 SMP Tue Jun 9 16:48:23 PDT 2015 x86_64 x86_64 x86_64 GNU/Linux",
"python_ver": "Python 2.6.6",
"release": "RedHat Linux Red Hat Enterprise Linux Server release 6.7 (Santiago)",
"rsync_ver": "rsync version 3.0.6 protocol version 30",
"ssh_ver": "OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013"
}
},
"servers": {
"CDUP01": {
"backups": {
"20160107T075815": {
"backup_id": "20160107T075815",
"backup_label": null,
"begin_offset": 32,
"begin_time": "Thu Jan 7 07:54:29 2016",
"begin_wal": "00000001000000000000001F",
"begin_xlog": "0/1F000020",
"config_file": "/var/lib/pgsql/data/postgresql.conf",
"deduplicated_size": null,
"end_offset": null,
"end_time": null,
"end_wal": null,
"end_xlog": null,
"error": null,
"hba_file": "/var/lib/pgsql/data/pg_hba.conf",
"ident_file": "/var/lib/pgsql/data/pg_ident.conf",
"included_files": null,
"mode": "default",
"pgdata": "/var/lib/pgsql/data",
"server_name": "CDUP01",
"size": null,
"status": "STARTED",
"tablespaces": null,
"timeline": 1,
"version": 80420
},
"20160315T113936": {
"backup_id": "20160315T113936",
"backup_label": null,
"begin_offset": 32,
"begin_time": "Tue Mar 15 11:34:57 2016",
"begin_wal": "000000010000000600000099",
"begin_xlog": "6/99000020",
"config_file": "/var/lib/pgsql/data/postgresql.conf",
"deduplicated_size": null,
"end_offset": null,
"end_time": null,
"end_wal": null,
"end_xlog": null,
"error": null,
"hba_file": "/var/lib/pgsql/data/pg_hba.conf",
"ident_file": "/var/lib/pgsql/data/pg_ident.conf",
"included_files": null,
"mode": "default",
"pgdata": "/var/lib/pgsql/data",
"server_name": "CDUP01",
"size": null,
"status": "STARTED",
"tablespaces": null,
"timeline": 1,
"version": 80420
},
"20160315T133355": {
"backup_id": "20160315T133355",
"backup_label": null,
"begin_offset": 32,
"begin_time": "Tue Mar 15 13:29:16 2016",
"begin_wal": "00000001000000060000009C",
"begin_xlog": "6/9C000020",
"config_file": "/var/lib/pgsql/data/postgresql.conf",
"deduplicated_size": null,
"end_offset": null,
"end_time": null,
"end_wal": null,
"end_xlog": null,
"error": "failure issuing start backup command (Cannot terminate exclusive backup. You might have to manually execute pg_stop_backup() on your PostgreSQL server)",
"hba_file": "/var/lib/pgsql/data/pg_hba.conf",
"ident_file": "/var/lib/pgsql/data/pg_ident.conf",
"included_files": null,
"mode": "default",
"pgdata": "/var/lib/pgsql/data",
"server_name": "CDUP01",
"size": null,
"status": "FAILED",
"tablespaces": null,
"timeline": 1,
"version": 80420
}
},
"config": {
"active": true,
"archiver": true,
"backup_directory": "/data/barman/CDUP01",
"backup_method": "rsync",
"backup_options": "exclusive_backup",
"bandwidth_limit": null,
"barman_home": "/data/barman",
"barman_lock_directory": "/data/barman",
"basebackup_retry_sleep": 30,
"basebackup_retry_times": 0,
"basebackups_directory": "/data/barman/CDUP01/base",
"compression": "gzip",
"conninfo": "host=cdup01.easyc.uni-c.dk user=postgres",
"custom_compression_filter": null,
"custom_decompression_filter": null,
"description": "CDUP01 TEST",
"disabled": false,
"errors_directory": "/data/barman/CDUP01/errors",
"immediate_checkpoint": false,
"incoming_wals_directory": "/data/barman/CDUP01/incoming",
"last_backup_maximum_age": null,
"minimum_redundancy": 0,
"msg_list": [],
"name": "CDUP01",
"network_compression": false,
"path_prefix": null,
"post_archive_retry_script": null,
"post_archive_script": null,
"post_backup_retry_script": null,
"post_backup_script": null,
"pre_archive_retry_script": null,
"pre_archive_script": null,
"pre_backup_retry_script": null,
"pre_backup_script": null,
"recovery_options": "",
"retention_policy": "window 2 d",
"retention_policy_mode": "auto",
"reuse_backup": null,
"ssh_command": "ssh post...@cdup01.easyc.uni-c.dk",
"streaming_archiver": false,
"streaming_conninfo": "host=cdup01.easyc.uni-c.dk user=postgres",
"streaming_wals_directory": "/data/barman/CDUP01/streaming",
"tablespace_bandwidth_limit": null,
"wal_retention_policy": "simple-wal 2 d",
"wals_directory": "/data/barman/CDUP01/wals"
},
"status": {
"archive_command": "rsync -a %p bar...@psql01.stil.dk:/data/barman/CDUP01/incoming",
"archive_mode": "on",
"config_file": "/var/lib/pgsql/data/postgresql.conf",
"current_size": 23836219.0,
"current_xlog": "0000000100000006000000B4",
"data_directory": "/var/lib/pgsql/data",
"hba_file": "/var/lib/pgsql/data/pg_hba.conf",
"ident_file": "/var/lib/pgsql/data/pg_ident.conf",
"last_archived_wal": null,
"pgespresso_installed": false,
"server_txt_version": "8.4.20"
},
"system_info": {
"kernel_ver": "Linux cdup01.easyc.uni-c.dk 2.6.39-400.215.2.el6uek.x86_64 #1 SMP Fri Jun 6 12:51:44 PDT 2014 x86_64 x86_64 x86_64 GNU/Linux",
"python_ver": "Python 2.6.6",
"release": "RedHat Linux Red Hat Enterprise Linux Server release 6.5 (Santiago)",
"rsync_ver": "rsync version 3.0.6 protocol version 30",
"ssh_ver": "OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013"
}
},
"CDUP02": {
"backups": {
"20151228T134927": {
"backup_id": "20151228T134927",
"backup_label": null,
"begin_offset": null,
"begin_time": null,
"begin_wal": null,
"begin_xlog": null,
"config_file": "/var/lib/pgsql/data/postgresql.conf",
"deduplicated_size": null,
"end_offset": null,
"end_time": null,
"end_wal": null,
"end_xlog": null,
"error": "failure issuing start backup command (pg_start_backup(): a backup is already in progress)",
"hba_file": "/var/lib/pgsql/data/pg_hba.conf",
"ident_file": "/var/lib/pgsql/data/pg_ident.conf",
"included_files": null,
"mode": "default",
"pgdata": "/var/lib/pgsql/data",
"server_name": "CDUP02",
"size": null,
"status": "FAILED",
"tablespaces": null,
"timeline": null,
"version": 80420
},
"20160325T104632": {
"backup_id": "20160325T104632",
"backup_label": null,
"begin_offset": 32,
"begin_time": "Fri Mar 25 10:46:32 2016",
"begin_wal": "000000010000000100000043",
"begin_xlog": "1/43000020",
"config_file": "/var/lib/pgsql/data/postgresql.conf",
"deduplicated_size": null,
"end_offset": null,
"end_time": null,
"end_wal": null,
"end_xlog": null,
"error": null,
"hba_file": "/var/lib/pgsql/data/pg_hba.conf",
"ident_file": "/var/lib/pgsql/data/pg_ident.conf",
"included_files": null,
"mode": "default",
"pgdata": "/var/lib/pgsql/data",
"server_name": "CDUP02",
"size": null,
"status": "STARTED",
"tablespaces": null,
"timeline": 1,
"version": 80420
}
},
"config": {
"active": true,
"archiver": true,
"backup_directory": "/data/barman/CDUP02",
"backup_method": "rsync",
"backup_options": "exclusive_backup",
"bandwidth_limit": null,
"barman_home": "/data/barman",
"barman_lock_directory": "/data/barman",
"basebackup_retry_sleep": 30,
"basebackup_retry_times": 0,
"basebackups_directory": "/data/barman/CDUP02/base",
"compression": "gzip",
"conninfo": "host=cdup02.easyc.uni-c.dk user=postgres",
"custom_compression_filter": null,
"custom_decompression_filter": null,
"description": "CDUP02 PROD",
"disabled": false,
"errors_directory": "/data/barman/CDUP02/errors",
"immediate_checkpoint": false,
"incoming_wals_directory": "/data/barman/CDUP02/incoming",
"last_backup_maximum_age": null,
"minimum_redundancy": 0,
"msg_list": [],
"name": "CDUP02",
"network_compression": false,
"path_prefix": null,
"post_archive_retry_script": null,
"post_archive_script": null,
"post_backup_retry_script": null,
"post_backup_script": null,
"pre_archive_retry_script": null,
"pre_archive_script": null,
"pre_backup_retry_script": null,
"pre_backup_script": null,
"recovery_options": "",
"retention_policy": "window 1 w",
"retention_policy_mode": "auto",
"reuse_backup": null,
"ssh_command": "ssh post...@cdup02.easyc.uni-c.dk",
"streaming_archiver": false,
"streaming_conninfo": "host=cdup02.easyc.uni-c.dk user=postgres",
"streaming_wals_directory": "/data/barman/CDUP02/streaming",
"tablespace_bandwidth_limit": null,
"wal_retention_policy": "simple-wal 1 w",
"wals_directory": "/data/barman/CDUP02/wals"
},
"status": {
"archive_command": "rsync -a %p bar...@psql01.stil.dk:/data/barman/CDUP02/incoming",
"archive_mode": "on",
"config_file": "/var/lib/pgsql/data/postgresql.conf",
"current_size": 36881883.0,
"current_xlog": "000000010000000100000044",
"data_directory": "/var/lib/pgsql/data",
"hba_file": "/var/lib/pgsql/data/pg_hba.conf",
"ident_file": "/var/lib/pgsql/data/pg_ident.conf",
"last_archived_wal": "000000010000000100000043.00000020.backup",
"pgespresso_installed": false,
"server_txt_version": "8.4.20"
},
"system_info": {
"kernel_ver": "Linux cdup02.easyc.uni-c.dk 2.6.39-400.215.2.el6uek.x86_64 #1 SMP Fri Jun 6 12:51:44 PDT 2014 x86_64 x86_64 x86_64 GNU/Linux",
"python_ver": "Python 2.6.6",
"release": "RedHat Linux Red Hat Enterprise Linux Server release 6.5 (Santiago)",
"rsync_ver": "rsync version 3.0.6 protocol version 30",
"ssh_ver": "OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013"
}
},
"CDUP03": {
"backups": {
"20160318T125416": {
"backup_id": "20160318T125416",
"backup_label": null,
"begin_offset": null,
"begin_time": null,
"begin_wal": null,
"begin_xlog": null,
"config_file": "/var/lib/pgsql/data/postgresql.conf",
"deduplicated_size": null,
"end_offset": null,
"end_time": null,
"end_wal": null,
"end_xlog": null,
"error": "failure issuing start backup command (pg_start_backup(): server closed the connection unexpectedly)",
"hba_file": "/var/lib/pgsql/data/pg_hba.conf",
"ident_file": "/var/lib/pgsql/data/pg_ident.conf",
"included_files": null,
"mode": "default",
"pgdata": "/var/lib/pgsql/data",
"server_name": "CDUP03",
"size": null,
"status": "FAILED",
"tablespaces": null,
"timeline": null,
"version": 80420
}
},
"config": {
"active": true,
"archiver": true,
"backup_directory": "/data/barman/CDUP03",
"backup_method": "rsync",
"backup_options": "exclusive_backup",
"bandwidth_limit": null,
"barman_home": "/data/barman",
"barman_lock_directory": "/data/barman",
"basebackup_retry_sleep": 30,
"basebackup_retry_times": 0,
"basebackups_directory": "/data/barman/CDUP03/base",
"compression": "gzip",
"conninfo": "host=195.231.251.156 user=postgres",
"custom_compression_filter": null,
"custom_decompression_filter": null,
"description": "CDUP PRE-PROD",
"disabled": false,
"errors_directory": "/data/barman/CDUP03/errors",
"immediate_checkpoint": false,
"incoming_wals_directory": "/data/barman/CDUP03/incoming",
"last_backup_maximum_age": null,
"minimum_redundancy": 0,
"msg_list": [],
"name": "CDUP03",
"network_compression": false,
"path_prefix": null,
"post_archive_retry_script": null,
"post_archive_script": null,
"post_backup_retry_script": null,
"post_backup_script": null,
"pre_archive_retry_script": null,
"pre_archive_script": null,
"pre_backup_retry_script": null,
"pre_backup_script": null,
"recovery_options": "",
"retention_policy": "window 1 w",
"retention_policy_mode": "auto",
"reuse_backup": null,
"ssh_command": "ssh post...@195.231.251.156",
"streaming_archiver": false,
"streaming_conninfo": "host=195.231.251.156 user=postgres",
"streaming_wals_directory": "/data/barman/CDUP03/streaming",
"tablespace_bandwidth_limit": null,
"wal_retention_policy": "simple-wal 1 w",
"wals_directory": "/data/barman/CDUP03/wals"
},
"status": {
"archive_command": "rsync -a %p bar...@psql01.stil.dk:/data/barman/CDUP03/incoming",
"archive_mode": "on",
"config_file": "/var/lib/pgsql/data/postgresql.conf",
"current_size": 27670467.0,
"current_xlog": "00000001000000070000009E",
"data_directory": "/var/lib/pgsql/data",
"hba_file": "/var/lib/pgsql/data/pg_hba.conf",
"ident_file": "/var/lib/pgsql/data/pg_ident.conf",
"last_archived_wal": "00000001000000070000009D",
"pgespresso_installed": false,
"server_txt_version": "8.4.20"
},
"system_info": {
"kernel_ver": "Linux cdup03.easyc.uni-c.dk 2.6.39-400.215.2.el6uek.x86_64 #1 SMP Fri Jun 6 12:51:44 PDT 2014 x86_64 x86_64 x86_64 GNU/Linux",
"python_ver": "Python 2.6.6",
"release": "RedHat Linux Red Hat Enterprise Linux Server release 6.5 (Santiago)",
"rsync_ver": "rsync version 3.0.6 protocol version 30",
"ssh_ver": "OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013"
}
},
"OrdblindetestTEST": {
"backups": {},
"config": {
"active": true,
"archiver": true,
"backup_directory": "/data/barman/OrdblindetestTEST",
"backup_method": "rsync",
"backup_options": "exclusive_backup",
"bandwidth_limit": null,
"barman_home": "/data/barman",
"barman_lock_directory": "/data/barman",
"basebackup_retry_sleep": 30,
"basebackup_retry_times": 0,
"basebackups_directory": "/data/barman/OrdblindetestTEST/base",
"compression": "gzip",
"conninfo": "host=test.ordblindetest.nu user=postgres",
"custom_compression_filter": null,
"custom_decompression_filter": null,
"description": "Ordblindetest TEST",
"disabled": false,
"errors_directory": "/data/barman/OrdblindetestTEST/errors",
"immediate_checkpoint": false,
"incoming_wals_directory": "/data/barman/OrdblindetestTEST/incoming",
"last_backup_maximum_age": null,
"minimum_redundancy": 0,
"msg_list": [],
"name": "OrdblindetestTEST",
"network_compression": false,
"path_prefix": null,
"post_archive_retry_script": null,
"post_archive_script": null,
"post_backup_retry_script": null,
"post_backup_script": null,
"pre_archive_retry_script": null,
"pre_archive_script": null,
"pre_backup_retry_script": null,
"pre_backup_script": null,
"recovery_options": "",
"retention_policy": "window 1 w",
"retention_policy_mode": "auto",
"reuse_backup": null,
"ssh_command": "ssh post...@test.ordblindetest.nu",
"streaming_archiver": false,
"streaming_conninfo": "host=test.ordblindetest.nu user=postgres",
"streaming_wals_directory": "/data/barman/OrdblindetestTEST/streaming",
"tablespace_bandwidth_limit": null,
"wal_retention_policy": "simple-wal 1 w",
"wals_directory": "/data/barman/OrdblindetestTEST/wals"
},
"status": {
"archive_command": null,
"archive_mode": null,
"current_xlog": null,
"data_directory": null,
"pgespresso_installed": null,
"server_txt_version": null
},
"system_info": {
"kernel_ver": "Linux test01.ordblindetest.nu 3.8.13-26.2.4.el6uek.x86_64 #2 SMP Fri May 9 11:36:29 PDT 2014 x86_64 x86_64 x86_64 GNU/Linux",
"python_ver": "Python 2.6.6",
"release": "LSB Version:\t:base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch\nDistributor ID:\tOracleServer\nDescription:\tOracle Linux Server release 6.5\nRelease:\t6.5\nCodename:\tn/a",
"rsync_ver": "rsync version 3.0.6 protocol version 30",
"ssh_ver": "OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013"
}
}
}
}
---------------------------------------------------
Reply all
Reply to author
Forward
0 new messages