Binary CNN in caffe/ristretto

[Mauricio C]

Hi!

I'm working with ristretto cause i'm researching on differents types of quantization techniques. About implemented binary CNN ( https://arxiv.org/abs/1602.02830 ). Is this possible to do  modifing ristretto code or using Python API?
If somebody has a clue about a solution to this I'll be greatful, because I'm starting to desing a solution.

Thanks a lot!

Best regards!


Mauricio

[pmg...@ucdavis.edu]

Hi Mauricio,

Thanks for your question about implementing binary networks in Caffe. Ristretto supports fixed point simulation and is not well-suited for very small bit-width representations (such as one bit). Ristretto contains re-implementations of the most important layer types in fixed point. The way Ristretto works is it just swaps floating point layers with fixed point layers. The gradient computation and weight updates are left unchanged.

For binary networks, however, you will have to change both the layer computation as well as the way the weight gradients get computed and applied. Moreover you will have to include a batch-norm layer which can handle the binary inputs and outputs.

The paper you mentioned has open-source code which could give you a good starting point. Good luck with the implementation!

Best,

Philipp