When using backup_compression, "Forcing PostgreSQLConnection cleanup during process shutdown" message is posted to barman.log file

[SteveB]

Hello,

We are experimenting with backup compression.  As soon as we set the backup_compression* parameters, we notice the following message is written to the barman.log file when barman cron is executed:

  barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down.

Does anyone know what might be causing the message?

Here are some the of the relevant environment settings:

  Barman 3.4 (installed on the same host as a Postgres 14.7 database)
  RHEL 9.1
  backup_method = postgres
  backup_compression = gzip
  backup_compression_format = tar
  backup_compression_location = client

We can successfully perform a backup to local storage with backup_compression enabled, and we see the backup size is 3.4 MiB for the most recent backup, compared to 25.6 MiB for all previous backups, which were taken without compression:

$ barman list-backups TESTDB2
TESTDB2 20230317T192605 - Fri Mar 17 19:26:05 2023 - Size: 3.4 MiB - WAL Size: 0 B
TESTDB2 20230317T145243 - Fri Mar 17 14:52:44 2023 - Size: 25.6 MiB - WAL Size: 51.1 KiB
TESTDB2 20230316T193002 - Thu Mar 16 19:30:03 2023 - Size: 25.6 MiB - WAL Size: 112.7 KiB
TESTDB2 20230315T193001 - Wed Mar 15 19:30:03 2023 - Size: 25.6 MiB - WAL Size: 64.4 KiB
TESTDB2 20230315T182601 - Wed Mar 15 18:26:03 2023 - Size: 25.6 MiB - WAL Size: 48.6 KiB

Also, we discovered another issue when running with backup_compression at the same time as using a hook script to replicate the backup to an S3 bucket -- the backup would hang trying to execute the hook script.  I will post a separate message for that.  In the above scenario, no hook scripts were enabled.

Thanks in advance for any insight you can provide.

Best regards,
Steve

[Mike Wallace]

Hi Steve,

> barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down.

This message occurs when a PostgreSQL connection is not closed
properly during Barman's execution and instead gets picked up by an
exit handler which ensures all connections are closed when Barman
terminates [1]. There is a known bug where the connection used for
validating compression settings is left open [2] and a fix for this
will be included in the next Barman release.

> Also, we discovered another issue when running with backup_compression at the same time as using a hook script to replicate the backup to an S3 bucket -- the backup would hang trying to execute the hook script. I will post a separate message for that. In the above scenario, no hook scripts were enabled.

barman-cloud-backup is unable to deal with the compressed base backup
format, so its use as a hook script with `backup_compression` and
`backup_compression_format = tar` is not currently supported. We'll
update the documentation so this limitation is explained (currently it
is not mentioned at all).

Best regards,

Mike

[1]: https://github.com/EnterpriseDB/barman/blob/release/3.4.0/barman/postgres.py#L75-L86
[2]: https://github.com/EnterpriseDB/barman/issues/719

> --
> --
> You received this message because you are subscribed to the "Barman for PostgreSQL" group.
> To post to this group, send email to pgba...@googlegroups.com
> To unsubscribe from this group, send email to
> pgbarman+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/pgbarman?hl=en?hl=en-GB
>
> ---
> You received this message because you are subscribed to the Google Groups "Barman, Backup and Recovery Manager for PostgreSQL" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pgbarman+u...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/pgbarman/d26071a8-edd2-40eb-9a06-ab8cf65ba983n%40googlegroups.com.

[SteveB]

Hi Mike,

Thanks very much for clarifying, and glad to know the warning message does not indicate anything to be concerned about.

Regarding the barman-cloud-backup limitation, is there any format of backup compression that can be replicated to cloud storage via hook script?

If not, I suppose a copy of compressed backups can at least be replicated to off-site non-cloud storage using passive geo-redundancy.

Best regards,

Steve

[Mike Wallace]

Hi Steve,

The `backup_compression_format` options map to `pg_basebackup`'s
`--format` options [1], so the only other available option is `plain`
- this is only useful when used with `backup_compression_location =
server` in order to reduce the amount of traffic over the network
during the backup, since the format on disk will be uncompressed.

A passive Barman server can indeed sync a compressed backup so that
option is available as a means of replicating a compressed backup to
offsite storage.

Best regards,

Mike

[1]: https://www.postgresql.org/docs/current/app-pgbasebackup.html

> To view this discussion on the web, visit https://groups.google.com/d/msgid/pgbarman/998aa2fe-49df-4923-a4d8-53365592ff65n%40googlegroups.com.