Re: zfs send compression query

[Bob Friesenhahn]

Using 'lzop' rather than 'pigz' is a better choice for CPU efficiency.
It offers more speed with less CPU use.

Bob
--
Bob Friesenhahn
bfri...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
_______________________________________________
freeb...@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "freebsd-fs-...@freebsd.org"

[John]

On Wed, Jan 06, 2016 at 05:01:44PM -0800, Sean Chittenden wrote:
>
>
>--
>Sean Chittenden
>
>
>> On Jan 6, 2016, at 15:35, John <freebs...@potato.growveg.org> wrote:
>>
>> $ zfs send -R vms/1...@2016-01-06.2122 | pigz -9c | ssh backup@[remote_ip] \
>> pigz -dc | zfs recv -Fdvu storage/snapshots

The problem I'm getting with that compressor (and others) is that it
bails like this:

send from @ to vms/1...@2016-01-07.0648 estimated size is 32.1G
total estimated size is 32.1G
cannot receive: specified fs (storage/snapshots) does not exist
Pseudo-terminal will not be allocated because stdin is not a terminal.
TIME SENT SNAPSHOT
-bash: line 1: syntax error near unexpected token |'

[then get a load of binary garbled output and then a broken pipe error]

zfs send works fine if I don't try to compress the stream:

$ zfs send -Rv vms/1...@2016-01-07.0648 | ssh backup@[redacted] zfs recv -Fdvu storage/snapshots
send from @ to vms/1...@2016-01-07.0648 estimated size is 32.1G
total estimated size is 32.1G
TIME SENT SNAPSHOT
receiving full stream of vms/1...@2016-01-07.0648 into
storage/snapshots/1...@2016-01-07.0648
07:26:44 300K vms/1...@2016-01-07.0648
07:26:45 1.92M vms/1...@2016-01-07.0648
07:26:46 3.42M vms/1...@2016-01-07.0648

I'm guessing I'm doing something wrong with the pipes, but not sure
how to solve it...
--
John

[John]

On Thu, Jan 07, 2016 at 12:05:36AM -0800, Sean Chittenden wrote:
>Bonus points:
>
>$ SNAP_SIZE=`zfs send -nvP "vms/1...@2016-01-07.0648" | tail -n 1 | awk '{print $2}'`
>$ zfs send -Rv vms/1...@2016-01-07.0648 | pv -r -a -b -t -e -s "${SNAP_SIZE}" -B 512m | pigz -9c | ssh backup@[redacted] "pigz -dc | zfs recv -Fdvu storage/snapshots"
>
>Useful for getting an ETA. -sc

Thanks for that. What do you think of other streaming compressors like
xz / lzrzip / pbzip2 / bzip2 / zpaq / pixz ?