Regarding Solver.prototxt

[Ashutosh Singla]

Hi,
I am new to Caffe. I would like to have some information about the following parameters from solver.prototxt file.
test_iter
test_interval
display
average_loss
snapshot.

It would be good if I have more information about this particular file.

[Oscar Beijbom]

On Tuesday, March 15, 2016 at 8:34:15 AM UTC-7, Ashutosh Singla wrote:

Hi,
I am new to Caffe. I would like to have some information about the following parameters from solver.prototxt file.

Hi, please see below. Hope this helps.

 

test_iter: number of iterations to run over the test-set
test_interval: interval between runs over the test-set
display: interval between displyaing the training-loss.
average_loss: For output purposes, the loss can be averaged across this many iterations. It's simply an average filter.
snapshot.: How often should caffe save a snapshot of the solverstate and model?

It would be good if I have more information about this particular file.

cheers 

[Ashutosh Singla]

Thanks a lot for the information.

[Jan]

You can find info like this in the proto file defining these parameters: https://github.com/BVLC/caffe/blob/master/src/caffe/proto/caffe.proto.

Jan

[Ashutosh Singla]

I have some questions regarding the solver.proto file:

I am training GoogLeNet model with 3k images for training and 1k for validation. I am wondering how can I improve the performance of my model. Below there are few parameters in the solver file which we can play with.

test_iter: 40
test_interval: 200
test_initialization: false
display: 200
average_loss: 40
base_lr: 0.01
lr_policy: "poly"
power: 0.5
max_iter: 10000
momentum: 0.9
weight_decay: 0.0002

Could you please explain me what does test_initialization, and weight_decay means?

Does the test_iter, and test_interval affects the performance of the training model? 

I know the meaning of test_iter, and test_interval

test_iter:  Validation set size / Validation batch size

Test_interval: How often would you like to get this estimation?

[Jan]

Look at the above referenced proto to find to out what they mean. For example, https://github.com/BVLC/caffe/blob/master/src/caffe/proto/caffe.proto#L144-146 explains brief but precisely what test_initialization does. No, test_iter and test_interval do not influence training performance in any way. weight_decay is a simple method of regularization, look into research literature or https://en.wikipedia.org/wiki/Regularization_(mathematics) to see how it works. It basically penalizes large weights in the layers with the ultimate goal of improving generalization and work against overfitting.

Jan