Use FastLZ for real-time compression

[Selso Liberado]

Hello !

AS fastLZ was referenced on stackoverflow for embbeded dev I did a try on it yesterday.

First I really like the smplest integration I could see : a pair of file, no configure or complicated script to handle fore having the source file to compile.

I tested 6pack on a 450 Mb containing many zéro data and it is extremely fast on a core2quad : 8,35 Mb gotten in less than 1 second

The problem a got is the API : I shall give the whole buffer in one call, and this complicate the streaming data.

For example LZ4 has 2 call to respectively add data to compress to the existing compressed buffer, and to finish the frame.

Such API is interesting as I want to transmit on USB data stored on the flash from my STM32 Nucleo board with the following scenarion :

• the frame has a fixed max size : 4 k.
• I must send data size 4k => I use compression.
• After initialization I set 3 k to compress (maybe this chunck data si not so much compressible and the compressor added a header)
• I then add the remaining data size in frame to compress
• and so on...
• I finalize the compression.

[Ariya Hidayat]

Hi Selso,

Although FastLZ doesn't offer frame/block API, it should be relatively simple to implement it yourself (on top of the existing API).

You can have a fixed-size input block (e.g. 4K) dan a variable-size output block. Just make sure to have a block metadata information (e.g. block header) to store information such as the compressed block size, checksum if necessary, etc.

Have fun with the project!

Best regards,

--
--
To post to this group, send email to fas...@googlegroups.com
To unsubscribe from this group, send email to fastlz-un...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/fastlz?hl=en
---
You received this message because you are subscribed to the Google Groups "fastlz" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fastlz+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fastlz/08168370-3e3f-4569-ba60-7268cf8e91a5%40googlegroups.com.

--

Ariya Hidayat, https://ariya.io

[Selso Liberado]

I readed twice but at least I understood :)

My problem was that the client side had to uncompress the frame's blocks would not know about each block size. But actually I just have to add a header as you said, with the block size...

But I still wonder what would be the size overhead because of repetead "header block" in frame, is it the HASH_SIZE table ?

I wonder if LZ4 do such thing or just keep one header.

I'll check about it...