Finetuning finetuning_flickr gives KeyError: Loss

[Ashish Bansal]

HI!

I am trying to use ILSVRC model and fine tune it for my purposes. I  have about 2000 images with some clothing attributes (6 classes):

men, 0

plaid, 1

shirts, 2

women, 3

woolen, 4

jacket, 5

Each image may have one or more such tags or classes attached to it. I modified the solved and train_val files from the base provided by the fintetune_flicker_style example and created a copy.  I have attached my model files. 

The train file looks like so:

/home/dev/Workspace/caffe/data/clothing/images/1451fd5a-9dab-4321-a5f8-081464ac5c11.jpg 0

/home/dev/Workspace/caffe/data/clothing/images/1451fd5a-9dab-4321-a5f8-081464ac5c11.jpg 1

/home/dev/Workspace/caffe/data/clothing/images/fdaa53c8-3f8a-487b-bf3a-b2406a34a3cf.jpg 0

/home/dev/Workspace/caffe/data/clothing/images/fdaa53c8-3f8a-487b-bf3a-b2406a34a3cf.jpg 1

/home/dev/Workspace/caffe/data/clothing/images/fdaa53c8-3f8a-487b-bf3a-b2406a34a3cf.jpg 2

Test file looks similar. I run the training using the iPython notebook provided. Specifically, my code looks like:

niter = 200

# losses will also be stored in the log

train_loss = np.zeros(niter)

scratch_train_loss = np.zeros(niter)

#  NO GPU :-(

# caffe.set_device(0)

# caffe.set_mode_gpu()

# We create a solver that fine-tunes from a previously trained network.

solver = caffe.SGDSolver('models/clothing/solver.prototxt')

solver.net.copy_from('models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel')

# For reference, we also create a solver that does no finetuning.

#scratch_solver = caffe.SGDSolver('models/clothing/solver.prototxt')

# We run the solver for niter times, and record the training loss.

for it in range(niter):

    solver.step(1)  # SGD by Caffe

    #scratch_solver.step(1)

    # store the train loss

    train_loss[it] = solver.net.blobs['loss'].data

    #scratch_train_loss[it] = scratch_solver.net.blobs['loss'].data

    if it % 10 == 0:

        print 'iter %d, finetune_loss=%f, scratch_loss=%f' % (it, train_loss[it], 0.0) #scratch_train_loss[it])

print 'done'

I dont have a GPU. So, I am trying to train on CPU.

As this is to get me familiar with this model, I removed the scratch solver just to see if this will run. It doesnt. This is what I get:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-2-0cc492e29598> in <module>()
     19     #scratch_solver.step(1)
     20     # store the train loss
---> 21     train_loss[it] = solver.net.blobs['loss'].data
     22     #scratch_train_loss[it] = scratch_solver.net.blobs['loss'].data
     23     if it % 10 == 0:

KeyError: 'loss'

What I see in the last few lines of the log of my notebook server is this:

....
I0122 12:49:02.382803 30248 net.cpp:163] norm2 needs backward computation.

I0122 12:49:02.382808 30248 net.cpp:163] pool2 needs backward computation.

I0122 12:49:02.382813 30248 net.cpp:163] relu2 needs backward computation.

I0122 12:49:02.382817 30248 net.cpp:163] conv2 needs backward computation.

I0122 12:49:02.382822 30248 net.cpp:163] norm1 needs backward computation.

I0122 12:49:02.382827 30248 net.cpp:163] pool1 needs backward computation.

I0122 12:49:02.383031 30248 net.cpp:163] relu1 needs backward computation.

I0122 12:49:02.383040 30248 net.cpp:163] conv1 needs backward computation.

I0122 12:49:02.383045 30248 net.cpp:165] label_data_1_split does not need backward computation.

I0122 12:49:02.383050 30248 net.cpp:165] data does not need backward computation.

I0122 12:49:02.383055 30248 net.cpp:201] This network produces output accuracy

I0122 12:49:02.383076 30248 net.cpp:446] Collecting Learning Rate and Weight Decay.

I0122 12:49:02.383085 30248 net.cpp:213] Network initialization done.

I0122 12:49:02.383090 30248 net.cpp:214] Memory required for data: 343011208

I0122 12:49:02.383229 30248 solver.cpp:42] Solver scaffolding done.

E0122 12:50:15.734585 30248 upgrade_proto.cpp:609] Attempting to upgrade input file specified using deprecated transformation parameters: models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel

I0122 12:50:15.735247 30248 upgrade_proto.cpp:612] Successfully upgraded file specified using deprecated data transformation parameters.

E0122 12:50:15.735581 30248 upgrade_proto.cpp:614] Note that future Caffe releases will only support transform_param messages for transformation fields.

E0122 12:50:15.735934 30248 upgrade_proto.cpp:618] Attempting to upgrade input file specified using deprecated V1LayerParameter: models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel

I0122 12:50:33.200902 30248 upgrade_proto.cpp:626] Successfully upgraded file specified using deprecated V1LayerParameter

I0122 12:50:33.381424 30248 solver.cpp:266] Iteration 0, Testing net (#0)

I0122 12:56:05.978937 30248 solver.cpp:315]     Test net output #0: accuracy = 0.1686

I0122 12:56:34.444236 30248 solver.cpp:189] Iteration 0, loss = 2.32055

I0122 12:56:34.444691 30248 solver.cpp:470] Iteration 0, lr = 0.001

Any ideas what is happening? Any guidance?