HiFiC Target BPP

[Ashan Perera]

Hello,

Thank you for releasing an open source HiFiC trainer. I am trying to train at custom target bitrates but the actual bitrate doesn't seem to change. In "configs.py," I tried two runs changing the target in the "mselpips" and "hific" configs, one run with 0.07 and another with 0.0001. Everything else was left at defaults or values described in the README (coco2014 dataset, 1M iterations for each model, etc.). Training ran on a NVIDIA A100. For both target bitrates, when evaluating on coco2014 I got an average BPP of 0.12-0.14. Is this a minimum bound for HiFiC or have I been specifying the target incorrectly? I would appreciate any clarification!

[Fabian Mentzer]

Hi, I'm not sure what's happening. How do the training curves look? two things come to mind:

a) We struggled to go below a certain minimum rate (but that was lower than 0.12 IIRC). You could try also reducing the number of channels in the bottleneck.

b) the bitrate is optimized on the training set. It could be that the evaluation set is somehow "harder". But since you are also training on coco I'm not sure about this theory

[George Toderici]

Something else to keep in mind is that when you train for a "bpp", that's not a universal bpp. It really is an average across a set of patches. Once you take that model and apply it it won't produce that specific bpp, but rather something "in that range". What the model is really trained for i sa specific RD tradeoff which will be more or less the same.

Coco is also TERRIBLE for training on. Sorry about that. It's my fault for making it the default. I just didn't know what other dataset to use. I would suggest switching to something less compressed. The images in COCO have very low quality, so your compression model will be misinformed about the real distribution of images. What you need are high quality images (imagine JPEG quality level 90+, whereas in COCO, it's way less than that).

George

--
You received this message because you are subscribed to the Google Groups "tensorflow-compression" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tensorflow-compre...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tensorflow-compression/7d5b138b-9001-47d7-8a9c-da2fccf9597cn%40googlegroups.com.

[Johannes Ballé]

The other code in TFC uses the CLIC dataset via TFDS. That might be an option here?

To view this discussion on the web visit https://groups.google.com/d/msgid/tensorflow-compression/CAFWCW%3Dg_YPEPJE0s-F_1FTpvMxjmg-B8UCjJdLt62O10eNh84A%40mail.gmail.com.

[George Toderici]

I'm unclear how much training data is needed. I suspect CLIC might be too small to properly train HiFiC on it.

To view this discussion on the web visit https://groups.google.com/d/msgid/tensorflow-compression/CAN4xYSAZVkUrVkf%2BM-%3DXL1pr%2BdwTBkfgGHhtQ05tdBig6uKfug%40mail.gmail.com.

[Ashan Perera]

Thank you everyone for the advice! Attached are the training curves I got with a 0.0001 target BPP. It's good to hear that it's possible for the BPP to go lower. than what I got The problems with the COCO dataset makes sense - I will try training with another high quality dataset if I can find one bigger than CLIC. 

[Fabian Mentzer]

Which loss is the `hific_loss.svg`? Gan loss? Looks reasonable to me.

Let us know if you make any progress with other datasets.

You received this message because you are subscribed to a topic in the Google Groups "tensorflow-compression" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tensorflow-compression/2NIkrxPGUDc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tensorflow-compre...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tensorflow-compression/77b8df8d-f3d0-47f3-86de-18bd2befd9acn%40googlegroups.com.

[Ashan Perera]

That's correct, "hific_loss.svg" is the GAN loss. I'll be sure to update after trying other datasets!