Blosc compression detection

[Miha Vitorovič]

Hi all,

I'm looking at using Blosc (decompression) in an environment where multiple codecs can be used. The decompression should happen transparently (no user intervention), so I'm wondering if there is some way of detecting whether the data received was compressed by Blosc?

Looking at the "bolsc.h" the 'blosc_cbuffer_complib" function like a possible candidate, however the description says it always succeeds - which makes it appear that the input buffer is assumed to contain Blosc compressed data.

Thank you for your answer and best regards, Miha

[Francesc Alted]

Hi Miha,

Yes, `blosc_cbuffer_complib()` is assuming that the buffer is a Blosc one.  C-Blosc2 format adds a new frame format where you can do more strict checks on whether the chunk to decompress is an actual Blosc2 frame (see https://github.com/Blosc/c-blosc/blob/master/README_HEADER.rst) or not.  However, be aware that C-Blosc2 is not ready for production yet.

For your use case you may want to look into the Blosc1 header (https://github.com/Blosc/c-blosc/blob/master/README_HEADER.rst) and try to apply some heuristics from there on your received data.

Hope this helps 

--
You received this message because you are subscribed to the Google Groups "blosc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blosc+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/blosc/077241dc-3edf-4d9e-8afe-c4930d09b19e%40googlegroups.com.

--

Francesc Alted