Remote restore missing WALs

[Victor Honoritus]

I have a barman server (10.10.10.10) and postgres server (20.20.20.20), I'm trying to restore from barman to postgres server. In postgres server I have a demo table to which I'm inserting data every second. I'm simulating failure by sudo rm -rf /var/lib/pgsql/15/data on the postgres server and my goal is to restore from barman with minimal data loss. Issue is when I start the postgres server, I only see data from around time when backup was created, so I lost the data between when the backup was created (14:42:31) and when i deleted pgdata folder (14:45:20)

Barman version: 3.16.2 

Postgres version: 15 

14:42:31 

barman backup myserver 

Starting backup using postgres method for server myserver in /var/lib/barman/myserver/base/20251204T144231 

Backup start at LSN: 0/190046B0 (000000040000000000000019, 000046B0) 

Starting backup copy via pg_basebackup for 20251204T144231 

Copy done (time: less than one second) 

Finalising the backup. Backup size: 32.8 MiB 

Backup end at LSN: 0/1B000000 (00000004000000000000001A, 00000000) 

Backup completed (start time: 2025-12-04 14:42:31.636135, elapsed time: less than one second) 

Processing xlog segments from streaming for myserver (batch size: 2) 000000040000000000000019 00000004000000000000001A 

14:45:20 

rm -rf /var/lib/pgsql/15/data 

Then i do remote restore:

sudo -u barman barman restore --get-wal --remote-ssh-command "ssh post...@20.20.20.20" myserver latest /var/lib/pgsql/15/data 

Starting remote restore for server myserver using backup 20251204T144231 

Destination directory: /var/lib/pgsql/15/data 

Remote command: ssh post...@20.20.20.20 Copying the base backup. 

Generating recovery configuration 

Identify dangerous settings in destination directory. 

WARNING: 'get-wal' is in the specified 'recovery_options'. 

Before you start up the PostgreSQL server, please review the postgresql.auto.conf file inside the target directory. 

Make sure that 'restore_command' can be executed by the PostgreSQL user. 

Restore operation completed (start time: 2025-12-04 14:46:09.367263+01:00, elapsed time: 5 seconds) 

Your PostgreSQL server has been successfully prepared for recovery! 

When I check latest data in the table:

select * from demotable order by created_at desc limit 3; 

id | created_at ------+---------------------------- 

2817 | 2025-12-04 14:42:32.517993 

2816 | 2025-12-04 14:42:31.516351 

2815 | 2025-12-04 14:42:30.514661

barman check myserver 

Server myserver: PostgreSQL: OK 

superuser or standard user with backup privileges: OK 

PostgreSQL streaming: OK 

wal_level: OK 

replication slot: OK

 directories: OK

 retention policy settings: OK 

backup maximum age: OK (no last_backup_maximum_age provided) 

backup minimum size: OK (32.8 MiB) 

wal maximum age: OK (no last_wal_maximum_age provided) 

wal size: OK  (16.0 MiB) compression settings: OK 

failed backups: OK (there are 0 failed backups) 

minimum redundancy requirements: OK (have 6 non-incremental backups, expected at least 0) 

pg_basebackup: OK 

pg_basebackup compatible: OK 

pg_basebackup supports tablespaces mapping: OK 

systemid coherence: OK 

pg_receivexlog: OK 

pg_receivexlog compatible: OK 

receive-wal running: OK 

archiver errors: OK

 barman replication-status myserver 

Status of streaming clients for server 'myserver':

 Current LSN on master: 0/1C0000D8 

Number of streaming clients: 1 1. 

Async WAL streamer Application name: barman_receive_wal 

Sync stage : 3/3 

Remote write Communication : TCP/IP

 IP Address : 20.20.20.20 / Port: 47976 / Host: - User name : streaming_replicator Current state : streaming (async) 

Replication slot: barman WAL sender

 PID : 3146994 

Started at : 2025-12-04 15:03:20.902992+01:00 

Sent LSN : 0/1C0000D8 (diff: 0 B) 

Write LSN : 0/1C0000D8 (diff: 0 B)

 Flush LSN : 0/1C000000 (diff: -216 B) 

barman show-backup myserver latest 

Backup 20251204T144231: 

Server Name : myserver 

System Id : 7572509021004650203 

Status : DONE PostgreSQL 

Version : 150015 

PGDATA directory : /var/lib/pgsql/15/data 

Estimated Cluster Size : 32.3 MiB

 Server information: Checksums : off 

Base backup information: 

Backup Method : postgres 

Backup Type : full 

Backup Size : 32.8 MiB (48.8 MiB with WALs) 

WAL Size : 16.0 MiB 

Timeline : 4 

Begin WAL : 00000004000000000000001A

 End WAL : 00000004000000000000001A 

Number of WALs : 1 

Begin time : 2025-12-04 14:42:31.633652+01:00 

End time : 2025-12-04 14:42:32.483446+01:00

 Copy time : less than one second 

Estimated throughput : 40.1 MiB/s 

Begin Offset : 40 

End Offset : 0

 Begin LSN : 0/1A000028 

End LSN : 0/1B000000 

WAL information: 

Number of files : 1

 Disk usage : 16.0 MiB 

WAL rate : 39344.50/hour 

Last available : 00000004000000000000001B 

Reachable timelines : 5 

Catalog information: Retention Policy : VALID 

Previous Backup : 20251204T135804 

Next Backup : - (this is the latest base backup)

 WARNING: WAL information is inaccurate due to multiple timelines interacting with this backup 

barman config 

[myserver] 

streaming_archiver = on 

backup_method = postgres 

streaming_conninfo = host=20.20.20.20 user=streaming_replicator dbname=postgres slot_name = barman 

create_slot = auto 

conninfo = host=20.20.20.20 user=barman dbname=postgres 

path_prefix = /usr/pgsql-15/bin

Thank you!

[Martin Marques]

I believe the problem here is that the data you lost is in the partial file in the `streaming_directory`. That file doesn't get archived until it fills up or a switch-wal happens. I believe we have a ticket for an enhancement here. You can get around this by not using `get-wal` (then the partial WAL will be copied with any other WAL).

--
--
You received this message because you are subscribed to the "Barman for PostgreSQL" group.
To post to this group, send email to pgba...@googlegroups.com
To unsubscribe from this group, send email to
pgbarman+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/pgbarman?hl=en?hl=en-GB

---
You received this message because you are subscribed to the Google Groups "Barman, Backup and Recovery Manager for PostgreSQL" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pgbarman+u...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/pgbarman/893ab9f1-c00d-47d5-8714-b8e27d20d7f0n%40googlegroups.com.

[Martin Marques]

Hi,

On Mon, 12 Jan 2026 at 09:35, Martin Marques <martin....@enterprisedb.com> wrote:
>
> I believe the problem here is that the data you lost is in the partial file in the `streaming_directory`. That file doesn't get archived until it fills up or a switch-wal happens. I believe we have a ticket for an enhancement here. You can get around this by not using `get-wal` (then the partial WAL will be copied with any other WAL).

I wanted to send a small correction (I don't know why I mixed up when we consume the .partial file and when we don't).

The recovery command you shared should have provided a `-P` to the restore_command so it consumes the partial file.

I don't know if this is something you solved already, but if you haven't, please check the value of the `restore_command` in the postgres configuration of the restored node, and check if it has the `-P` option.

You can read about this here: https://docs.pgbarman.org/release/3.17.0/user_guide/recovery.html#limitations-of-partial-wal-files

Cheers,