Question regards to solver.test_nets[0] in notebook example with custom training loop

[Atena Nguyen]

Hello all, 

The custom training loop is written as follows (Link: Solving in Python with LeNet)

%%time
niter = 200
test_interval = 25
# losses will also be stored in the log
train_loss = zeros(niter)
test_acc = zeros(int(np.ceil(niter / test_interval)))
output = zeros((niter, 8, 10))

# the main solver loop
for it in range(niter):
    solver.step(1)  # SGD by Caffe
    
    # store the train loss
    train_loss[it] = solver.net.blobs['loss'].data
    
    # store the output on the first test batch
    # (start the forward pass at conv1 to avoid loading new data)
    solver.test_nets[0].forward(start='conv1')
    output[it] = solver.test_nets[0].blobs['score'].data[:8]
    
    # run a full test every so often
    # (Caffe can also do this for us and write to a log, but we show here
    #  how to do it directly in Python, where more complicated things are easier.)
    if it % test_interval == 0:
        print 'Iteration', it, 'testing...'
        correct = 0
        for test_it in range(100):
            solver.test_nets[0].forward()
            correct += sum(solver.test_nets[0].blobs['score'].data.argmax(1)
                           == solver.test_nets[0].blobs['label'].data)
        test_acc[it // test_interval] = correct / 1e4

I do not understand the for loop for test_it (highlighted in yellow color). From my understanding, the line

solver.test_nets[0].forward()

calculates the forward pass of the test_nets for one batch (the batch size is 100 in this example). Then 

solver.test_nets[0].blobs['score'].data.argmax(1)

predicts the output of the network for 100 input test which was compared with the label 

solver.test_nets[0].blobs['label'].data

to count how many inputs have been predicted correctly as 

 correct += sum(solver.test_nets[0].blobs['score'].data.argmax(1) == solver.test_nets[0].blobs['label'].data)

So, my question is: 

1. test_it is changing every iteration but not considered in any in-loop code. Why?

2. What happen for each iteration? Do this line

solver.test_nets[0].forward()

the forward pass for different mini-batch at different iteration?  and 

solver.test_nets[0].blobs['score'].data.argmax(1)

calculates the output for different mini-batch at different iteration? 

 

3. the test_it is in range (100), any particular reason for this? My initial guess is no. as they normalize corect to 1e4 which is the (no iter) 100 x 100(mini-batch size)

4. Is it always solver.test_nets[0]? how about test_nets[1] or so on? 

Thank you very much for your time, 

[Przemek D]

1. It could as well be for _ in range(100):, since the only purpose of this line is to say: "repeat the following 100 times". We don't need the index of the current iteration because...
2. net.forward() loads new data batch internally - at every iteration, new mini-batch is loaded.
3. MNIST dataset has 10k testing images. In caffe example, the LeNet for TEST phase has batch size of 100 - which means, we need to run (10k examples / 100 in a batch) = 100 batches in order to go through the entire testing set.