WAL compression and compression_filter

[MP]

Hello,

1] why is extension removed from compressed WAL file?  If i want manually decompress files, i need pipe them (or rename) in scripts == little more complexity (+ I need check configs for compression type)
2] if i set compression_filter = lz4, will it work ? As point 1], lz4 needs lz4 extension to decompress. How barman knows what extension is needed for decompressing? xlog.db dont store extension. Gzip and bzip2 support is nice, but they are slow, especialy for iteration compressing WALs.

Thank you

[MP]

update: compression_filter -> custom_compression_filter

What parameters are needed to set in custom_compression_filter and custom_decompression_filter for working with LZ4 ?

[Gabriele Bartolini]

Hi Martin,

the main goal of Barman is to simplify disaster recovery management
for any user, by hiding inner complexity. That is the reason why WAL
segments are hidden in the archive and extension is removed.

As far as lz4 compression is concerned, you can use custom
compression filters, which operate through standard input and output.
We decided to embed support for the two most common compression
algorithms: gzip and bzip2. This might change in the future, in case
we find users interested in that.

Cheers,
Gabriele
--
Gabriele Bartolini - 2ndQuadrant Italia
PostgreSQL Training, Services and Support
gabriele....@2ndQuadrant.it | www.2ndQuadrant.it

> --
> --
> You received this message because you are subscribed to the "Barman for
> PostgreSQL" group.
> To post to this group, send email to pgba...@googlegroups.com
> To unsubscribe from this group, send email to
> pgbarman+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/pgbarman?hl=en?hl=en-GB
>
> ---
> You received this message because you are subscribed to the Google Groups
> "Barman, Backup and Recovery Manager for PostgreSQL" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pgbarman+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Joseph Kregloh]

You can always setup the backup in a filesystem that has built in compression. For example have a ZFS filesystem with LZ4 compression enabled. This will be the easiest and even faster way to compress the WAL files. I have all my backups on a ZFS array with gzip compression enabled. That way it even compresses the backup files, not just the WAL files, and it is done on a filesystem level.

-Joseph