Skip to first unread message
CG
Jul 30, 2017, 4:36:01 PM7/30/17
to Caffe Users
I know this is the caffe-users group, but perhaps someone knows the answer to this question related to the SegNet implementation in Caffe.
SegNet has its own implementaton of the batch normalization layer. What this BN layer does is: during training uses the mini-batch mean and variance to normalize, scale and shift. -the normal way-
To test the network, a script (compute_bn_statistics.py) must be previously run that modifies the weights of the trained network by calculating the global mean and variance over the whole training set, and outputs a new caffemodel containing the new weights, so we could say that the global statistics calculation is done "offline".
During inference, the layer has a mode "INFERENCE" that does only the shift and scale.
If caffe has already a BatchNorm layer that does exactly the same, what do you think is the reason to compute the global statistics offline? So far it has only made things more complicated for multiple reasons, including, running extra scripts after training, and not being able to see the test accuracy/loss during training.
If anyone has worked with Caffe-SegNet before, do you have any idea on how to properly test during training time using the batchnorm layers?
I am thinking in replacing the SegNet BN layers for the BatchNorm + Scale of Caffe. I dont see any use of it besides saving some milliseconds not dealing with the global statistics during inference.
Thank you
SegNet has its own implementaton of the batch normalization layer. What this BN layer does is: during training uses the mini-batch mean and variance to normalize, scale and shift. -the normal way-
To test the network, a script (compute_bn_statistics.py) must be previously run that modifies the weights of the trained network by calculating the global mean and variance over the whole training set, and outputs a new caffemodel containing the new weights, so we could say that the global statistics calculation is done "offline".
During inference, the layer has a mode "INFERENCE" that does only the shift and scale.
If caffe has already a BatchNorm layer that does exactly the same, what do you think is the reason to compute the global statistics offline? So far it has only made things more complicated for multiple reasons, including, running extra scripts after training, and not being able to see the test accuracy/loss during training.
If anyone has worked with Caffe-SegNet before, do you have any idea on how to properly test during training time using the batchnorm layers?
I am thinking in replacing the SegNet BN layers for the BatchNorm + Scale of Caffe. I dont see any use of it besides saving some milliseconds not dealing with the global statistics during inference.
Thank you
Reply all
Reply to author
Forward
0 new messages