NoisyNormalMixture in ContinuousIndexedEntropyModel example

[mahadev prasad panda]

Hello,

I am trying to use a NoisyNormalMixture prior function in ContinuousIndexedEntropyModel for a Hyperprior Model.

I found the example for the NoisyLogisticMixture in the GitHub implementation and made some modification given below to fit my requirements.

```
tfc.ContinuousIndexedEntropyModel(
prior_fn=tfc.NoisyNormalMixture,
index_ranges=(64, 64, 64),
parameter_fns=dict(
loc=lambda i: i[..., 0:3],
scale=lambda i: i[…, 3:6],
weight=lambda i: tf.nn.softmax(i[..., 6:9]),
),
coding_rank=3,
channel_axis=-1,
)
```

This runs fine while training, but when I am trying to use this entropy model in testing, it keeps on running for days without throwing any error. Is there something wrong the way I am using the entropy model? Also, if there is any example of  use of NoisyNormalMixture in ContinuousIndexedEntropyModel, please let me know.

Thank you!

Best Regards

Mahadev

[Johannes Ballé]

Hi Mahadev,

I think you have run into one of the limitations of the current TFC implementation.

One problem is that the number of entries in the range coding table grows exponentially with the number of parameters you are sweeping over in the model. In your example, there are 64 x 64 x 64 = 262144 possible combinations of distribution parameters, which means there are over 250k different scalar distributions that need to be prepared for use in a range coder.

Not only is the range coding table going to be quite large (It is going to be a tensor with 262144 x K elements, where K is the average number of distinct probability mass values within each scalar distribution.), but our current code is also not designed for this large of a dimension.

So, it is likely that the code spends a lot of time in creating that range coding table. That said, once it has been created, the code should run.

We could use some help in speeding up this code. On the other hand, we are currently shifting our resources towards CoDeX (a JAX-based compression library), so any help we can get would be appreciated!

Best,

Johannes.

--
You received this message because you are subscribed to the Google Groups "tensorflow-compression" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tensorflow-compre...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tensorflow-compression/8a8cd75e-c384-490f-960e-398c97726b2bn%40googlegroups.com.

[mahadev prasad panda]

Hi  Johannes,

Thank you for your reply. 

Unfortunately, I am not pursuing the project anymore. So I am not able to provide any help regarding the speeding up of the code.

Regards,

Mahadev