Help on interpreting the log printed by caffe

[Lillian Liu]

Hello, 

I have a testing set of 27700 examples. 

I set "batch_size: 100" in test.prototxt . 

I set "test_iter: 277"  "test_interval: 1000"  in solver.prototxt. 

FYI, my test_net has three outputs including loss, accuracy and prob as shown below in test.prototxt:

#compute the loss

layer {

  name: "loss"

  type: "SoftmaxWithLoss"

  bottom: "score"

  bottom: "label"

  top: "loss"

}

#also output accuracy

layer {

  name: "accuracy"

  type: "Accuracy"

  bottom: "score"

  bottom: "label"

  top: "accuracy"

}

#also probability for each example and each class

layer {

  name: "prob"

  type: "Softmax"

  bottom: "score"

  top: "prob"

}

Then I looked at the stuff printed by Caffe and I am confused about the following part. So it seems to do a testing every 1000 training iterations as expected, and it prints out the following for iteration 2000:

I0509 09:15:36.000584 38819 solver.cpp:337] Iteration 2000, Testing net (#0)

I0509 09:16:32.002871 38819 solver.cpp:404]     Test net output #0: accuracy = 0.874115

I0509 09:16:32.002920 38819 solver.cpp:404]     Test net output #1: loss = 7.54223 (* 1 = 7.54223 loss)

I0509 09:16:32.002928 38819 solver.cpp:404]     Test net output #2: prob = 0.844104

I0509 09:16:32.002936 38819 solver.cpp:404]     Test net output #3: prob = 0.155896

....

....

I0509 09:16:32.004505 38819 solver.cpp:404]     Test net output #200: prob = 0.844409

I0509 09:16:32.004510 38819 solver.cpp:404]     Test net output #201: prob = 0.155591

I0509 09:16:32.425251 38819 solver.cpp:228] Iteration 2000, loss = 1.26618

I0509 09:16:32.425300 38819 solver.cpp:244]     Train net output #0: accuracy = 0.98

I0509 09:16:32.425315 38819 solver.cpp:244]     Train net output #1: loss = 1.26618 (* 1 = 1.26618 loss)

I0509 09:16:32.425328 38819 sgd_solver.cpp:106] Iteration 2000, lr = 8.72196e-06

Which means it prints out 200 = 2x100 probabilities for the testing phase. I have 2 output classes so that is 100 testing examples. But I though it was supposed to test test_iter * batch_size = 27700 examples (i.e., my whole testing set) every test_interval (1000 here) training iterations. Why there are only 100 showing here? 

Could you please help me out!

Thanks a lot!!!!

Lillian

[Jan]

This is how the solver runs a test phase: It feeds test_iter batches (of the size specified in your data layer) through the test net, records the values of the output blobs (which is every blob that is only used as a top, but never as a bottom) after every iteration, and averages the recorded values over all test_iter iterations. So the logging output you see should have the size of the output blob (which often depends on the batch_size for "regular" layer outputs, but are single values for loss and accuracy layer outputs), and the values printed are averages of the values at this position over all test batches.

Jan

[Lillian Liu]

Thank you so much!! That clears up my confusion!!

Lillian