Re: [caffe-users] What is 'moving_average_fraction_' in Batch Normalization?

[Jonathan R. Williford]

It doesn't affect the training. During validation and testing, the batch normalization needs to use a static transform. It does not use the mean and variance of the validation or testing batch, which could be of size one. Instead, it uses an estimate of mean and variance from the training. The estimate is calculated by taking a moving average.

See also:

http://caffe.berkeleyvision.org/doxygen/classcaffe_1_1BatchNormLayer.html#details

Jonathan

On Wed, Nov 16, 2016 at 9:26 AM, MKR <cryx...@gmail.com> wrote:

Hello,

Can anybody explain to me what 'moving_average_fraction_' parameter in a Batch Normalization layer does?
I implemented it in a ResNet.
layer {
  name: "bn2a_branch2b"
  type: "BatchNorm"
  bottom: "res2a_branch2b"
  top: "res2a_branch2b"
  include {
    phase: TRAIN
  }
  batch_norm_param {
    use_global_stats: false
    moving_average_fraction: 0.95 
  }
}
layer {
  name: "bn2a_branch2b"
  type: "BatchNorm"
  bottom: "res2a_branch2b"
  top: "res2a_branch2b"
  include {
    phase: TEST
  }
  batch_norm_param {
    use_global_stats: true
   moving_average_fraction: 0.95 
  }
}

Thanks!

--
You received this message because you are subscribed to the Google Groups "Caffe Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to caffe-users+unsubscribe@googlegroups.com.
To post to this group, send email to caffe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/caffe-users/0a01ee23-89f2-49fe-af52-5b13afbc63c8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.