How is bpp more than packed_bpp?

[Mohana]

Hi,

I trained the ms2020 code on some dummy dataset, and tried to compress an image using the trained model. The 'total_bpp' that is returned by run method, which is supposed to be the Shannon Information of compressed data ( a "lower bound"), is always higher than the actual bpp (packed_bpp) which includes the overhead of tensor shapes. I don't understand why this is happening, is it because I didn't train the model enough or am I wrong in thinking that actual bpp can only be greater than or equal to Shannon info, not lesser?

Eg output for kodim03.png (Kodak dataset) :

Information content in bpp: 1.1236 (supposed to be a lower bound)

Actual bits per pixel: 1.1023 (is lower than the lower bound)

Thanks a lot!

[Johannes Ballé]

Hi Mohana,

you aren't doing anything wrong – we have observed this occasionally as well. I'll look into this when I get a chance and update the documentation. In the grand scheme of things, what the documentation says should be correct ,but I suspect it may have to do with the fact that the range coding tables need to be discretized themselves, so the probability model used for range coding is actually slightly different from the one used for estimating the Shannon entropy.

Best,

Johannes.

--
You received this message because you are subscribed to the Google Groups "tensorflow-compression" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tensorflow-compre...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tensorflow-compression/142f555f-d399-47b0-8123-1e2f0f4a5b54n%40googlegroups.com.

[Johannes Ballé]

I checked the math, and this is actually because these bounds only hold in the limit, for an infinite amount of data. In the latest version of TFC, we have explicit unit tests for them:

https://github.com/tensorflow/compression/blob/master/tensorflow_compression/python/entropy_models/continuous_batched_test.py#L112