Alexnet quantized weights doesn't match with 8-bit representation

[akz001]

Hello,

Assumptions:

• Default caffe settings

The precision reported by caffe-ristretto was 8bit, but i found the values extracted from caffemodel were of much higher precision which cannot be represented by 8 bits. (The weights are extracted as we extract from a normal caffemodel file created by caffe in single precision format)

For example:

The lowest value from layer#1 weights(conv1) was 0.0000015 and this can't be represented using 8 bits.

The quantization parameters read as given below-

quantization_param {

    bw_layer_in: 8

    bw_layer_out: 8

    bw_params: 8

    fl_layer_in: 0

    fl_layer_out: -3

    fl_params: 8} 

So my question is, how ristretto represent this small number with lower precision as 8 bits OR  Is there anything I am missing here ?

Thanks

[alexey.ch...@gmail.com]

Same question here.
I use the Ristretto version of SqueezeNet: when I read the weights for the first layer from the corresponding Ristretto-SqueezeNet caffemodel file, the values are:

>>w.ravel()[0:5]
array([-0.00391747, -0.00511208, -0.03623973, -0.04713646, -0.05958042], dtype=float32)

In the prototxt file it is said that
  quantization_param {
...
    bw_params: 8
...
    fl_params: 7
  }

So if fl_params is 7, it means that by muliplying by 2**7 I should get integer numbers, right?
However, all I get this
>>w.ravel()[0:5]*128
array([-0.50143605, -0.65434664, -4.6386857 , -6.03346682, -7.62629318], dtype=float32)

These are obviously not integers.
Or am I missing something here?

[Philipp Gysel]

Hi guys,

Thanks for your question! You are correct in that the weights in *.caffemodel are in higher precision. Ristretto quantizes the weights on the fly for each batch. If it helps, have a look at the foward path implementation of Ristretto layers: The weights are copied to a second blob and quantized in there.

Best,

Philipp

[paki...@gmail.com]

Hi Philipp,

why do you quantize on the fly instead of doiing it when the layer is loaded?