Barman Copies the data , But the remote server is not recovering

[Savita Pandey]

Please help 

I recovered remote server , but restarting postgres on remote server is not working, as in the databases are not being recovered on the remote server 

Used this command 

time barman recover --target-time "2022-02-23 05:35:56.254051+00:00" --remote-ssh-command "ssh postgres@test-ip” test 20220223T045333 /mnt/data/postgresql/10/main/data $PGDATA

Your PostgreSQL server has been successfully prepared for recovery!

real    16m31.532s
user    8m34.357s
sys     3m56.060s

--

Thanks

Savita Pandey

4695378034

[Michael Wallace]

Firstly, can you confirm exactly what state PostgreSQL is in after restarting on the remote server. Does it fail to start? Does it start up successfully but the expected data is not available? Is any data available at all or are you seeing an empty PostgreSQL instance?

Secondly, could you please share the following:

* The logs from the barman recover command.

* The logs from postgres when it is restarted on the remote server.

* The output of the `barman diagnose` command from your barman server (be careful to remove any sensitive information before sharing).

Thanks,

Mike

[Savita Pandey]

Hi Michael. 

Thank you for response , 

This is one problem that I am seeing , I don't see the recovery logs on the barman server . I see all backup logs but no logs are being generated during recovery, Or postgresql recovery logs . I have looked into all possible logging directories. I am not sure whats happening 

Barman diagnose is attached 

[Siddhartha Gurijala]

Savita,

By recovery log, i believe what mike was referring to was the info printed out when you ran the recovery command. 

Example : 

Starting backup using rsync-exclusive method for server test-pgserver in /var/lib/barman/test-pgserver/base/20220122T200998

Backup start at LSN: 342/79000028 (000000010000034200000079, 00000028)

Starting backup copy via rsync/SSH for 20220122T200998 (20 jobs)

Copy done (time: 45 seconds)

Asking PostgreSQL server to finalize the backup.

Backup size: 243.7 GiB. Actual size on disk: 12.1 GiB (-96.41% deduplication ratio).

Backup end at LSN: 342/794FF9D0 (000000010000034200000079, 004FF9D0)

Backup completed (start time: 2022-02-25 12:09:09.846159, elapsed time: 1 minute, 2 seconds)

Such log information would help us understand if the recovery was done in the right way or not.

Thanks & Regards, Sid

[Savita Pandey]

Here is the log that is see 

barman@prod-usw2-cs-main-dbbackups:~$ time barman recover --target-time "2022-02-25 03:51:21.494358+00:00" --remote-ssh-command "ssh postgres@remote-server-ip" dbname 20220225T033001  /mnt/data/postgresql/10/main
Starting remote restore for server halberd_streaming using backup 20220225T033001
Destination directory: /mnt/data/postgresql/10/main
Remote command: ssh postgres@remote-server-ip
Doing PITR. Recovery target time: '2022-02-25 03:51:21.494358+00:00'
Copying the base backup.

Your PostgreSQL server has been successfully prepared for recovery!
real  16m31.532s
user  8m34.357s
sys  3m56.060s

Thanks

Savita Pandey

--
--
You received this message because you are subscribed to the "Barman for PostgreSQL" group.
To post to this group, send email to pgba...@googlegroups.com
To unsubscribe from this group, send email to
pgbarman+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/pgbarman?hl=en?hl=en-GB

---
You received this message because you are subscribed to the Google Groups "Barman, Backup and Recovery Manager for PostgreSQL" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pgbarman+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pgbarman/97d0cad1-babc-40d7-9f92-4742bd5eae89n%40googlegroups.com.

--

Thanks

Savita Pandey

469537803 

[Siddhartha Gurijala]

Why do you have time before the recover command?

per official documentation here is the syntax of the recover command.

The recover command is used to recover a whole server after a backup is executed using the backup command.

This is achieved issuing a command like the following:

barman@backup$ barman recover <server_name> <backup_id> /path/to/recover/dir

IMPORTANT: Do not issue a recover command using a target data directory where a PostgreSQL instance is running. In that case, remember to stop it before issuing the recovery. This applies also to tablespace directories.

https://docs.pgbarman.org/release/2.18/

[Savita Pandey]

Hi 

Thank you for your response , the time command just gives me the exact time taken to recover. But i have tried without time as well but still the issue still persists.

Thank you

savita pandey

To view this discussion on the web, visit https://groups.google.com/d/msgid/pgbarman/e562c92d-6773-4c07-9509-f65022fd5f0an%40googlegroups.com.

[Michael Wallace]

The `barman recover` output looks ok so we really need to see the PostgreSQL logs when it starts up on the recovered PGDATA directory. If you check your postgresql.conf file at /etc/postgresql/10/main/postgresql.conf then you should be able to find where PostgreSQL is sending its logs.

To view this discussion on the web, visit https://groups.google.com/d/msgid/pgbarman/CAMAE62%2BLKyDuweBiYeW6Upjd76qd3JBL9KKaVP8fZP71vkbVXA%40mail.gmail.com.

[Savita Pandey]

Hi 

So i finally recreated the issue ,

barman@prod-usw2-cs-main-dbbackups:/mnt/data/barman/bmshard1/config$ barman recover --target-time "2022-02-26 05:03:10.894436+00:00" --remote-ssh-command "ssh postgres@host-ip" db 20220226T050001  /mnt/data/postgresql/10/main/
Starting remote restore for server bmshard1 using backup 20220226T050001
Destination directory: /mnt/data/postgresql/10/main/
Remote command: ssh post...@10.5.84.61
Doing PITR. Recovery target time: '2022-02-26 05:03:10.894436+00:00'
Copying the base backup.

Copying required WAL segments.
Generating recovery configuration
Identify dangerous settings in destination directory.

WARNING
The following configuration files have not been saved during backup, hence they have not been restored.
You need to manually restore them in order to start the recovered PostgreSQL instance:

    postgresql.conf
    pg_hba.conf
    pg_ident.conf

Recovery completed (start time: 2022-02-26 20:33:53.416241, elapsed time: 10 minutes, 41 seconds)
Your PostgreSQL server has been successfully prepared for recovery!

I start my recovered postgresql server like
/usr/lib/postgresql/10/bin/pg_ctl -D /mnt/data/postgresql/10/main -l logfile start
waiting for server to start.... stopped waiting
pg_ctl: could not start server
Examine the log output.
PostgreSQL logs
postgres: could not access the server configuration file "/mnt/data/postgresql/10/main/postgresql.conf": No such file or directory

I have postgresql.conf in below directory
conf.d	environment  pg_ctl.conf  pg_hba.conf  pg_ident.conf  postgresql.conf  start.conf
/etc/postgresql/10/main

my data directory is in different directory
/mnt/data/postgresql/10/main/
PG_VERSION    current_logfiles	pg_logical    pg_serial     pg_subtrans  pg_xact		      recovery_conf.source
backup_label  global		pg_multixact  pg_snapshots  pg_tblspc	 postgresql.auto.conf
barman_wal    pg_commit_ts	pg_notify     pg_stat	    pg_twophase  postgresql.auto.conf.origin
base	      pg_dynshmem	pg_replslot   pg_stat_tmp   pg_wal	 recovery.conf

To view this discussion on the web, visit https://groups.google.com/d/msgid/pgbarman/CAGxDL9KJMWRxWMXA_UjVG9xN48ZdKeN3%2BQFcqMOUABeP%3DWG6sg%40mail.gmail.com.

[Siddhartha Gurijala]

Looks like the reason your server is not starting is because of missing conf files in data directory that said you need to verify what the path is set for your postgresql data directory. 

To view this discussion on the web, visit https://groups.google.com/d/msgid/pgbarman/CAMAE62J3MkaiLZ%2BKjXhx03C29_sefh%2B3By45jdenMc6wfg_L4g%40mail.gmail.com.

--

Siddhartha Reddy Gurijala

Senior Data Analyst
Nodal Exchange LLC
http://www.nodalexchange.com

w: (703) 962-9866

Confidentiality Notice: This communication may
contain confidential information. If you are
not the intended recipient, please return this
message to the sender and delete all copies
hereof without disclosing or using the
information in this e-mail.

[Luca Ferrari]

On Sun, Feb 27, 2022 at 11:45 AM Siddhartha Gurijala
<guri...@nodalexchange.com> wrote:
>
> Looks like the reason your server is not starting is because of missing conf files in data directory that said you need to verify what the path is set for your postgresql data directory.

or the files are there but the permissions are wrong and thus the
operating system "postgres" (I suspect) cannot access them.

Luca

[Savita Pandey]

Thank you Luca .Figured it out. It was something to do with how that server was configured, tested the recovery on another server . But i started the recovery command yesterday, and Recovery is still not complete 

What can be the reasons for recovery taking that time 

--
--
You received this message because you are subscribed to the "Barman for PostgreSQL" group.
To post to this group, send email to pgba...@googlegroups.com
To unsubscribe from this group, send email to
pgbarman+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/pgbarman?hl=en?hl=en-GB

---
You received this message because you are subscribed to the Google Groups "Barman, Backup and Recovery Manager for PostgreSQL" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pgbarman+u...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/pgbarman/CAKoxK%2B6q5hDwW_GAgEAwrdaHuBmPr94pGFx76wz2LvZd6H1wEA%40mail.gmail.com.