Barman not connecting to Postgres

762 views
Skip to first unread message

Salman Khan

unread,
Apr 18, 2017, 7:14:50 AM4/18/17
to Barman, Backup and Recovery Manager for PostgreSQL
Hi,

I have setup Barman 1.5 over my Postgres 9.3. Both are on the same server but Barman won't connect to the Postgres however, the same configuration works fine otherwise. Note that Barman user has superuser rights.
I couldn't find .pgpass file under Barman so I created one and made entries as addressed in the below link but of no use.


Server main:

        PostgreSQL: FAILED

        directories: OK

        retention policy settings: OK

        backup maximum age: OK (no last_backup_maximum_age provided)

        compression settings: OK

        minimum redundancy requirements: FAILED (have 0 backups, expected at least 1)

        ssh: OK (PostgreSQL server)

        not in recovery: OK



Barman logs:


2017-04-18 15:50:58,042 [27303] barman.cli DEBUG: Initialised Barman version 1.5.1 (config: /etc/barman.conf, args: {'command': 'check', 'server_name': ['main'], 'format': 'console', 'debug': False, 'quiet': False, 'nagios': False})

2017-04-18 15:50:58,051 [27303] barman.server DEBUG: Error retrieving PostgreSQL setting 'data_directory': must be superuser to examine "data_directory"

 

2017-04-18 15:50:58,052 [27303] barman.server DEBUG: Error retrieving PostgreSQL setting 'wal_level': current transaction is aborted, commands ignored until end of transaction block

 

2017-04-18 15:50:58,052 [27303] barman.server DEBUG: Error retrieving PostgreSQL version: current transaction is aborted, commands ignored until end of transaction block

 

2017-04-18 15:50:58,053 [27303] barman.server DEBUG: Error retrieving pgespresso information: current transaction is aborted, commands ignored until end of transaction block

 

2017-04-18 15:50:58,054 [27303] barman.server DEBUG: Error calling pg_is_in_recovery() function: current transaction is aborted, commands ignored until end of transaction block

 

2017-04-18 15:50:58,054 [27303] barman.server DEBUG: Error retrieving current xlog: current transaction is aborted, commands ignored until end of transaction block

 

2017-04-18 15:50:58,055 [27303] barman.server DEBUG: Error retrieving PostgreSQL configuration files location: current transaction is aborted, commands ignored until end of transaction block

 

2017-04-18 15:50:58,056 [27303] barman.server DEBUG: Error retrieving PostgreSQL setting 'data_directory': current transaction is aborted, commands ignored until end of transaction block

 

2017-04-18 15:50:58,056 [27303] barman.server ERROR: Check 'PostgreSQL' failed for server 'main'

2017-04-18 15:50:58,057 [27303] barman.server DEBUG: Check 'directories' succeeded for server 'main'

2017-04-18 15:50:58,057 [27303] barman.server DEBUG: Check 'retention policy settings' succeeded for server 'main'

2017-04-18 15:50:58,057 [27303] barman.server DEBUG: Check 'backup maximum age' succeeded for server 'main'

2017-04-18 15:50:58,057 [27303] barman.server DEBUG: Check 'compression settings' succeeded for server 'main'

2017-04-18 15:50:58,058 [27303] barman.server ERROR: Check 'minimum redundancy requirements' failed for server 'main'

2017-04-18 15:50:58,058 [27303] barman.command_wrappers DEBUG: Command: ['ssh', 'post...@127.0.0.1', '-o', 'BatchMode=yes', '-o', 'StrictHostKeyChecking=no', 'true']

2017-04-18 15:50:58,174 [27303] barman.command_wrappers DEBUG: Command return code: 0

2017-04-18 15:50:58,174 [27303] barman.command_wrappers DEBUG: Command stdout:

2017-04-18 15:50:58,175 [27303] barman.command_wrappers DEBUG: Command stderr:

2017-04-18 15:50:58,175 [27303] barman.server DEBUG: Check 'ssh' succeeded for server 'main'

2017-04-18 15:50:58,181 [27303] barman.server DEBUG: Check 'not in recovery' succeeded for server 'main'


Diagnose Output:


{

    "global": {

        "config": {

            "barman_home": "/home/barman",

            "barman_user": "barman",

            "errors_list": [],

            "log_file": "/home/barman/barman.log",

            "log_level": "DEBUG"

        },

        "system_info": {

            "barman_ver": "1.5.1",

            "kernel_ver": "Linux A0310SAPIGWEdgeAx01.bss.telenor.com.pk 2.6.32-431.el6.x86_64 #1 SMP Sun Nov 10 22:19:54 EST 2013 x86_64 x86_64 x86_64 GNU/Linux",

            "python_ver": "Python 2.6.6",

            "release": "RedHat Linux Red Hat Enterprise Linux Server release 6.5 (Santiago)",

            "rsync_ver": "rsync  version 3.0.6  protocol version 30",

            "ssh_ver": "OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010"

        }

    },

    "servers": {

        "main": {

            "backups": {},

            "config": {

                "active": true,

                "backup_directory": "/home/barman/main",

                "backup_options": "exclusive_backup",

                "bandwidth_limit": null,

                "barman_home": "/home/barman",

                "barman_lock_directory": "/home/barman",

                "basebackup_retry_sleep": 30,

                "basebackup_retry_times": 0,

                "basebackups_directory": "/home/barman/main/base",

                "compression": null,

                "conninfo": "host=127.0.0.1 user=postgres password=postgres dbname=postgres port=5432",

                "copy_method": "rsync",

                "custom_compression_filter": null,

                "custom_decompression_filter": null,

                "description": null,

                "disabled": false,

                "immediate_checkpoint": false,

                "incoming_wals_directory": "/home/barman/main/incoming",

                "last_backup_maximum_age": null,

                "minimum_redundancy": 1,

                "msg_list": [],

                "name": "main",

                "network_compression": false,

                "post_archive_retry_script": null,

                "post_archive_script": null,

                "post_backup_retry_script": null,

                "post_backup_script": null,

                "pre_archive_retry_script": null,

                "pre_archive_script": null,

                "pre_backup_retry_script": null,

                "pre_backup_script": null,

                "recovery_options": "",

                "retention_policy": null,

                "retention_policy_mode": "auto",

                "reuse_backup": null,

                "ssh_command": "ssh post...@127.0.0.1",

                "tablespace_bandwidth_limit": null,

                "wal_retention_policy": "main",

                "wals_directory": "/home/barman/main/wals"

            },

            "status": {

                "archive_command": "(disabled)",

                "archive_mode": "off",

                "current_xlog": null,

                "data_directory": null,

                "last_archived_wal": null,

                "pgespresso_installed": false,

                "server_txt_version": null,

                "wal_level": null

            },

            "system_info": {

                "kernel_ver": "Linux A0310SAPIGWEdgeAx01.bss.telenor.com.pk 2.6.32-431.el6.x86_64 #1 SMP Sun Nov 10 22:19:54 EST 2013 x86_64 x86_64 x86_64 GNU/Linux",

                "python_ver": "Python 2.6.6",

                "release": "RedHat Linux Red Hat Enterprise Linux Server release 6.5 (Santiago)",

                "rsync_ver": "rsync  version 3.0.6  protocol version 30",

                "ssh_ver": "OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010"

            }

        }

    }

}



Any help on this would be much appreciated.


Regards,

Salman



Salman Khan

unread,
Apr 19, 2017, 7:38:35 AM4/19/17
to pgba...@googlegroups.com
I was able to fix this with a few article from this group. It had some permissions issue and setting acls resolved that.
However, now the backup is stuck at:

2017-04-19 16:27:05,519 [18083] barman.backup_executor INFO: Copy done.
2017-04-19 16:27:05,521 [18083] barman.backup_executor INFO: Asking PostgreSQL server to finalize the backup.

I have assigned all necessary permissions and checked configurations as per other topics here but to no success. 

Diagnose output:

{
    "global": {
        "config": {
            "barman_home": "/home/barman", 
            "barman_user": "barman", 
            "compression": "gzip", 
            "errors_list": [], 
            "log_file": "/home/barman/barman.log", 
            "log_level": "DEBUG"
        }, 
        "system_info": {
            "barman_ver": "1.5.1", 
            "kernel_ver": "Linux A0310SAPIGWEdgeAx01.bss.telenor.com.pk 2.6.32-431.el6.x86_64 #1 SMP Sun Nov 10 22:19:54 EST 2013 x86_64 x86_64 x86_64 GNU/Linux", 
            "python_ver": "Python 2.6.6", 
            "release": "RedHat Linux Red Hat Enterprise Linux Server release 6.5 (Santiago)", 
            "rsync_ver": "rsync  version 3.0.6  protocol version 30", 
            "ssh_ver": "OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010"
        }
    }, 
    "servers": {
        "main": {
            "backups": {
                "20170419T154900": {
                    "backup_id": "20170419T154900", 
                    "backup_label": null, 
                    "begin_offset": 40, 
                    "begin_time": "Wed Apr 19 10:49:00 2017", 
                    "begin_wal": "000000080000001800000066", 
                    "begin_xlog": "18/66000028", 
                    "config_file": "/opt/apigee/data/apigee-postgresql/pgdata/postgresql.conf", 
                    "deduplicated_size": null, 
                    "end_offset": null, 
                    "end_time": null, 
                    "end_wal": null, 
                    "end_xlog": null, 
                    "error": null, 
                    "hba_file": "/opt/apigee/data/apigee-postgresql/pgdata/pg_hba.conf", 
                    "ident_file": "/opt/apigee/data/apigee-postgresql/pgdata/pg_ident.conf", 
                    "included_files": null, 
                    "mode": "default", 
                    "pgdata": "/opt/apigee/data/apigee-postgresql/pgdata", 
                    "server_name": "main", 
                    "size": null, 
                    "status": "STARTED", 
                    "tablespaces": null, 
                    "timeline": 8, 
                    "version": 90310
                }, 
                "20170419T160603": {
                    "backup_id": "20170419T160603", 
                    "backup_label": null, 
                    "begin_offset": 40, 
                    "begin_time": "Wed Apr 19 11:06:03 2017", 
                    "begin_wal": "000000080000001800000068", 
                    "begin_xlog": "18/68000028", 
                    "config_file": "/opt/apigee/data/apigee-postgresql/pgdata/postgresql.conf", 
                    "deduplicated_size": null, 
                    "end_offset": null, 
                    "end_time": null, 
                    "end_wal": null, 
                    "end_xlog": null, 
                    "error": null, 
                    "hba_file": "/opt/apigee/data/apigee-postgresql/pgdata/pg_hba.conf", 
                    "ident_file": "/opt/apigee/data/apigee-postgresql/pgdata/pg_ident.conf", 
                    "included_files": null, 
                    "mode": "default", 
                    "pgdata": "/opt/apigee/data/apigee-postgresql/pgdata", 
                    "server_name": "main", 
                    "size": null, 
                    "status": "STARTED", 
                    "tablespaces": null, 
                    "timeline": 8, 
                    "version": 90310
                }, 
                "20170419T161117": {
                    "backup_id": "20170419T161117", 
                    "backup_label": null, 
                    "begin_offset": 40, 
                    "begin_time": "Wed Apr 19 11:11:17 2017", 
                    "begin_wal": "00000008000000180000006A", 
                    "begin_xlog": "18/6A000028", 
                    "config_file": "/opt/apigee/data/apigee-postgresql/pgdata/postgresql.conf", 
                    "deduplicated_size": null, 
                    "end_offset": null, 
                    "end_time": null, 
                    "end_wal": null, 
                    "end_xlog": null, 
                    "error": null, 
                    "hba_file": "/opt/apigee/data/apigee-postgresql/pgdata/pg_hba.conf", 
                    "ident_file": "/opt/apigee/data/apigee-postgresql/pgdata/pg_ident.conf", 
                    "included_files": null, 
                    "mode": "default", 
                    "pgdata": "/opt/apigee/data/apigee-postgresql/pgdata", 
                    "server_name": "main", 
                    "size": null, 
                    "status": "STARTED", 
                    "tablespaces": null, 
                    "timeline": 8, 
                    "version": 90310
                }, 
                "20170419T162538": {
                    "backup_id": "20170419T162538", 
                    "backup_label": null, 
                    "begin_offset": 40, 
                    "begin_time": "Wed Apr 19 11:25:38 2017", 
                    "begin_wal": "00000008000000180000006C", 
                    "begin_xlog": "18/6C000028", 
                    "config_file": "/opt/apigee/data/apigee-postgresql/pgdata/postgresql.conf", 
                    "deduplicated_size": null, 
                    "end_offset": null, 
                    "end_time": null, 
                    "end_wal": null, 
                    "end_xlog": null, 
                    "error": null, 
                    "hba_file": "/opt/apigee/data/apigee-postgresql/pgdata/pg_hba.conf", 
                    "ident_file": "/opt/apigee/data/apigee-postgresql/pgdata/pg_ident.conf", 
                    "included_files": null, 
                    "mode": "default", 
                    "pgdata": "/opt/apigee/data/apigee-postgresql/pgdata", 
                    "server_name": "main", 
                    "size": null, 
                    "status": "STARTED", 
                    "tablespaces": null, 
                    "timeline": 8, 
                    "version": 90310
                }
            }, 
            "config": {
                "active": true, 
                "backup_directory": "/home/barman/main", 
                "backup_options": "exclusive_backup", 
                "bandwidth_limit": null, 
                "barman_home": "/home/barman", 
                "barman_lock_directory": "/home/barman", 
                "basebackup_retry_sleep": 30, 
                "basebackup_retry_times": 0, 
                "basebackups_directory": "/home/barman/main/base", 
                "compression": "gzip", 
                "archive_command": "sudo rsync -a /opt/apigee/data/apigee-postgresql/pgdata bar...@127.0.0.1:/home/barman/main/incoming/%f", 
                "archive_mode": "on", 
                "config_file": "/opt/apigee/data/apigee-postgresql/pgdata/postgresql.conf", 
                "current_xlog": "00000008000000180000006D", 
                "data_directory": "/opt/apigee/data/apigee-postgresql/pgdata", 
                "hba_file": "/opt/apigee/data/apigee-postgresql/pgdata/pg_hba.conf", 
                "ident_file": "/opt/apigee/data/apigee-postgresql/pgdata/pg_ident.conf", 
                "last_archived_wal": "00000006.history", 
                "pgespresso_installed": false, 
                "server_txt_version": "9.3.10", 
                "wal_level": "hot_standby"
            }, 
            "system_info": {
                "kernel_ver": "Linux A0310SAPIGWEdgeAx01.bss.telenor.com.pk 2.6.32-431.el6.x86_64 #1 SMP Sun Nov 10 22:19:54 EST 2013 x86_64 x86_64 x86_64 GNU/Linux", 
                "python_ver": "Python 2.6.6", 
                "release": "RedHat Linux Red Hat Enterprise Linux Server release 6.5 (Santiago)", 
                "rsync_ver": "rsync  version 3.0.6  protocol version 30", 
                "ssh_ver": "OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010"
            }
        }
    }
}


Regards,
Salman

--
--
You received this message because you are subscribed to the "Barman for PostgreSQL" group.
To post to this group, send email to pgba...@googlegroups.com
To unsubscribe from this group, send email to
pgbarman+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/pgbarman?hl=en?hl=en-GB

---
You received this message because you are subscribed to a topic in the Google Groups "Barman, Backup and Recovery Manager for PostgreSQL" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pgbarman/DObNH4Qh5jc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pgbarman+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Gabriele Bartolini

unread,
Apr 19, 2017, 9:36:41 AM4/19/17
to pgba...@googlegroups.com
Hi Salman,

  the archive_command you are using is completely wrong. While Barman is there to help configuring PostgreSQL Disaster recovery, that does not prevent you from knowing the basics and fundamentals of PITR in PostgreSQL.

  Also, we are putting a lot of efforts in the documentation and I think that is pretty clear in explaining what needs to be done and all the steps you have to follow.

  Obviously it is not, as I imagine you have read it thoroughly. According to you, is there anything you think it should be improved in the documentation? If so, please feel free to send a pull request as even the documentation is open source.

Thanks,
Gabriele

P.S.: By the way, backing up locally is a massive SPOF. I hope you are aware of that.

--
 Gabriele Bartolini - 2ndQuadrant Italia - Director
 PostgreSQL Training, Services and Support
 gabriele....@2ndQuadrant.it | www.2ndQuadrant.it

You received this message because you are subscribed to the Google Groups "Barman, Backup and Recovery Manager for PostgreSQL" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pgbarman+unsubscribe@googlegroups.com.

Salman Khan

unread,
Apr 19, 2017, 11:49:36 PM4/19/17
to pgba...@googlegroups.com
Hi,

Thanks for the reply and pointing it out.
The efforts you have put in the tool and it's documentation are really appreciated. Keep up the good work.
I have a few points that I would like to take up. Have been working on barman for the past week now without a gap. I'll note them down and share.
Lastly I am aware of the SPOF but this is the staging environment. I'll use a separate server for production backups.

Regards, 
Salman

On Apr 19, 2017 6:36 PM, "Gabriele Bartolini" <gabriele....@2ndquadrant.it> wrote:
Hi Salman,

  the archive_command you are using is completely wrong. While Barman is there to help configuring PostgreSQL Disaster recovery, that does not prevent you from knowing the basics and fundamentals of PITR in PostgreSQL.

  Also, we are putting a lot of efforts in the documentation and I think that is pretty clear in explaining what needs to be done and all the steps you have to follow.

  Obviously it is not, as I imagine you have read it thoroughly. According to you, is there anything you think it should be improved in the documentation? If so, please feel free to send a pull request as even the documentation is open source.

Thanks,
Gabriele

P.S.: By the way, backing up locally is a massive SPOF. I hope you are aware of that.
--
 Gabriele Bartolini - 2ndQuadrant Italia - Director
 PostgreSQL Training, Services and Support
 gabriele.bartolini@2ndQuadrant.it | www.2ndQuadrant.it

Salman Khan

unread,
Apr 20, 2017, 5:11:43 AM4/20/17
to pgba...@googlegroups.com
Hi,

I have changed the archive_command but still stuck at the same point. "barman.backup_executor INFO: Asking PostgreSQL server to finalize the backup"

Barman Diagnose:

{
    "global": {
        "config": {
            "barman_home": "/home/barman", 
            "barman_user": "barman", 
            "compression": "gzip", 
            "errors_list": [], 
            "log_file": "/home/barman/barman.log", 
            "log_level": "DEBUG"
        }, 
        "system_info": {
            "barman_ver": "1.5.1", 
            "kernel_ver": "Linux A0310SAPIGWEdgeAx01.bss.telenor.com.pk 2.6.32-431.el6.x86_64 #1 SMP Sun Nov 10 22:19:54 EST 2013 x86_64 x86_64 x86_64 GNU/Linux", 
            "python_ver": "Python 2.6.6", 
            "release": "RedHat Linux Red Hat Enterprise Linux Server release 6.5 (Santiago)", 
            "rsync_ver": "rsync  version 3.0.6  protocol version 30", 
            "ssh_ver": "OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010"
        }
    }, 
    "servers": {
        "main": {
            "backups": {
                "20170420T135517": {
                    "backup_id": "20170420T135517", 
                    "backup_label": null, 
                    "begin_offset": 40, 
                    "begin_time": "Thu Apr 20 08:55:17 2017", 
                    "begin_wal": "00000008000000180000007A", 
                    "begin_xlog": "18/7A000028", 
                "archive_command": "rsync -a %p bar...@127.0.0.1:/home/barman/main/incoming/%f", 
                "archive_mode": "on", 
                "config_file": "/opt/apigee/data/apigee-postgresql/pgdata/postgresql.conf", 
                "current_xlog": "00000008000000180000007B", 
                "data_directory": "/opt/apigee/data/apigee-postgresql/pgdata", 
                "hba_file": "/opt/apigee/data/apigee-postgresql/pgdata/pg_hba.conf", 
                "ident_file": "/opt/apigee/data/apigee-postgresql/pgdata/pg_ident.conf", 
                "last_archived_wal": "00000006.history", 
                "pgespresso_installed": false, 
                "server_txt_version": "9.3.10", 
                "wal_level": "archive"
            }, 
            "system_info": {
                "kernel_ver": "Linux A0310SAPIGWEdgeAx01.bss.telenor.com.pk 2.6.32-431.el6.x86_64 #1 SMP Sun Nov 10 22:19:54 EST 2013 x86_64 x86_64 x86_64 GNU/Linux", 
                "python_ver": "Python 2.6.6", 
                "release": "RedHat Linux Red Hat Enterprise Linux Server release 6.5 (Santiago)", 
                "rsync_ver": "rsync  version 3.0.6  protocol version 30", 
                "ssh_ver": "OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010"
            }
        }
    }
}
On Thu, Apr 20, 2017 at 8:49 AM, Salman Khan <salman....@gmail.com> wrote:
Hi,

Thanks for the reply and pointing it out.
The efforts you have put in the tool and it's documentation are really appreciated. Keep up the good work.
I have a few points that I would like to take up. Have been working on barman for the past week now without a gap. I'll note them down and share.
Lastly I am aware of the SPOF but this is the staging environment. I'll use a separate server for production backups.

Regards, 
Salman

Gabriele Bartolini

unread,
Apr 20, 2017, 12:55:38 PM4/20/17
to pgba...@googlegroups.com
Hi Salman,

  Can I ask you why you installed Barman 1.5.1 instead of 2.1? Also, have you followed the instructions regarding setting up WAL archiving?

Cheers,
Gabriele

--
 Gabriele Bartolini - 2ndQuadrant Italia - Director
 PostgreSQL Training, Services and Support
 gabriele....@2ndQuadrant.it | www.2ndQuadrant.it

Salman Khan

unread,
May 8, 2017, 4:05:21 AM5/8/17
to pgba...@googlegroups.com
Hi,

I tried barman 2.1 but WAL streaming had some issue and due to time constraints I went with 1.5.
I was able to setup barman and take backups on staging. The issue was with permissions on the pgdata directory. I set an ACL on the pgdata giving access to barman.
Now I have to replicate this on production. It has a Master Slave configuration of Postgres. Do I need to install barman on both servers or can I have the barman tool installed separately? What would be the configuration in this case for barman 1.5? As 2.1 supports different conf files but there is only one for barman 1.5.

Regards,
Salman

On Thu, Apr 20, 2017 at 9:55 PM, Gabriele Bartolini <gabriele....@2ndquadrant.it> wrote:
Hi Salman,

  Can I ask you why you installed Barman 1.5.1 instead of 2.1? Also, have you followed the instructions regarding setting up WAL archiving?

Cheers,
Gabriele
--
 Gabriele Bartolini - 2ndQuadrant Italia - Director
 PostgreSQL Training, Services and Support
 gabriele.bartolini@2ndQuadrant.it | www.2ndQuadrant.it

Reply all
Reply to author
Forward
0 new messages