Testing GoogLeNet with Pycaffe from lmdb

[Caleb Belth]

I'm trying to test each image in my test lmdb one by one through pycaffe so that I can calculate any statistics I want to. I have the following script to do this. However, when I test using the "caffe test" command line interface (which tests with batches) I get around 70% accuracy on my test set, but I'm getting about 1% accuracy using the script below. Does anyone know what might be different? Could someone point me to the Caffe code that does the testing for the command line interface so I can compare? I'm using the standard https://github.com/BVLC/caffe/tree/master/models/bvlc_googlenet train_val.prototxt and deploy.prototxt with only the num_output for classes changed because I don't have 1,000 classes.

lmdb = lmdb.open(lmdb_path)

lmdb_txn = lmdb.begin()

lmdb_cursor = lmdb_txn.cursor()

net = caffe.Net(model, weights, caffe.TEST)

transformer = caffe.io.Transformer({'data': net.blobs['data'].data.shape})

transformer.set_transpose('data', (2,0,1))

transformer.set_channel_swap('data', (2,1,0))

transformer.set_raw_scale('data', 255.0)

total = 0

correct = 0

count = 0

for key, value in lmdb_cursor:

    total += 1

    datum = caffe.proto.caffe_pb2.Datum()

    datum.ParseFromString(value)

    label = int(datum.label)

    image = caffe.io.datum_to_array(datum)

    image = image.astype(np.uint8)

    image = transformer.preprocess('data', image)

    out = net.forward_all(data=np.asarray([image]))

    plabel = int(out['prob'][0].argmax(axis=0))

    if label == plabel:

correct += 1

    print "scored: " + str(total) + " actual: " + labels[label] + " predicted: " + labels[plabel] + " correct: " + str(correct) + " accuracy: " + str(float(correct)/float(total))

[Ajay Shekar]

I was running into a similar issue. If you subtracted the mean from each image during train time, you should make sure to do the same at test time. This article talks about it! Hope this helps.