LZ4F_compressBound result

[stei...@scionics.de]

auto  nbytes=LZ4F_compressBound(64<< 10,prefs);

Hi,

I am just looking at the LZ4F_compressBound function call (see above). I observed that if I set LZ4F_max256KB in the preferences (prefs), and call LZ4F_compressBound like in the above, the nbytes is always around 256kB and nothing near 64 kB as I would expect it. Is this is a feature or a bug?

Best,

Peter

[stei...@scionics.de]

I should add that this is with lz4 1.8.0 on fc26.

[Cyan]

Right.

LZ4F_compressBound() has to take into consideration any potential "internal caching" that might occur as a result of prior input.

Take an example with the following scenario : 

- set blocks to 256 KB
- send 128KB to compress => no compression is started, those 128KB are only stored internally.

- send 128KB - 1 bytes to compress => still no compression, internal input is filled with 256KB-1 bytes.
- send 64KB to compression => compression is performed on the first 256KB of data sent (including previous sends), the remaining 64KB - 1 bytes are stored for next block.

Therefore, LZ4F_compressBound() will answer that worst case is a non-compressible 256 KB block.

This will be the same answer for any amount of input data between 1 byte and 256 KB.

Starting with 256KB + 1 bytes, worst case becomes 2 non-compressible blocks.

And so on.

[Peter Steinbach]

Hi Cyan,

thanks again for such a prompt answer. Alright, just so I get the hang
of it: LZ4F_compressBound will always output the number of required
Bytes in approximate chunks of the configured block size. No
configuration of the block size in LZ4F_preferences_t will default to
64kB. Anything else will result in number of output bytes that follow
the blocksize.

I'd love it if this information would end up in the docs of
LZ4F_compressBound. It'd offer to send a PR, if needed.

Best,
P

[Cyan]

LZ4F_compressBound() documentation was updated

to include more details, as per this discussion :

https://github.com/lz4/lz4/blob/dev/lib/lz4frame.h#L252

hope this helps

[Peter Steinbach]

Excellent, that makes it very clear.
Peter

> --
> Vous recevez ce message, car vous êtes abonné au groupe Google Groupes
> "LZ4c".
> Pour vous désabonner de ce groupe et ne plus recevoir d'e-mails le
> concernant, envoyez un e-mail à l'adresse
> lz4c+uns...@googlegroups.com
> <mailto:lz4c+uns...@googlegroups.com>.
> Pour obtenir davantage d'options, consultez la page
> https://groups.google.com/d/optout.

--
Peter Steinbach, Dr. rer. nat.
Scientific Software Engineer, Scientific Computing Facility

Scionics Computer Innovation GmbH
Löscherstr. 16
01309 Dresden
Germany

phone +49 351 210 2882
fax +49 351 202 707 04
www.scionics.de

Sitz der Gesellschaft: Dresden (Main office)
Amtsgericht - Registergericht: Dresden HRB 20337 (Commercial Registry)
Ust-IdNr.: DE813263791 (VAT ID Number)
Geschäftsführer: John Duperon, Jeff Oegema (Managing Directors)