Trouble extracting feature using pycaffe

[Sam B.]

Hello,

I'm trying to write a test script to extract a feature from a single image. I've had a lot of trouble finding documentation, and I've pretty much pulled snippets of code from various things I have found in an effort to make this work. I know what I need to do is:

-Transpose the image using 'set_transpose('data', (2,0,1))'

-Swap channels from rgb to bgr, 'set_channel_swap('data', (2,1,0))'

-Subtract the mean. Note that the mean file I have been supplied with is a .binary and not a .protobinary.

Now I know that this works in the command-line interface, however I read that pycaffe requires pre-processing to be done in the python script rather than in the .prototxt file. If it makes any difference, here's the data layer from the .prototxt file which was used for the command line interface:

layer {
  name: "data"
  type: "ImageData"
  top: "data"
  top: "label"
  include {
    phase: TEST
  }
  transform_param {
    mirror: false
    crop_size: 96
    mean_file: "trainingMean.binary"
  }
  image_data_param {
    source: "Test.txt"
  batch_size: 1  
  }
}

Now my python script is:

net=caffe.Net('blobs-Test-Eval.prototxt', 'best_snapshot_iter_4553.caffemodel', caffe.TEST)
transformer = caffe.io.Transformer({'data': net.blobs['data'].data.shape})
transformer.set_transpose('data', (2,0,1))
transformer.set_channel_swap('data', (2,1,0))
transformer.set_mean('data', np.load('trainingMean.binary'))
im=np.array(Image.open('Test/D_4051.png'))
im_input = im[np.newaxis, np.newaxis, :, :]
net.blobs['data'].data[...] = transformer.preprocess('data', caffe.io.load_image('Test/D_4051.png'))
out=net.forward()
print out['fc8']

First, am I right to assume I need to delete the data layer from the prototxt file when I run it using the python interface?

I'm guessing I'm doing a lot of things wrong in my python script, however at this point the error I'm getting is:

Traceback (most recent call last):
  File "pyTest.py", line 23, in <module>
    transformer.set_mean('data', np.load('trainingMean.binary'))
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/lib/npyio.py", line 398, in load
    "Failed to interpret file %s as a pickle" % repr(file))
IOError: Failed to interpret file 'trainingMean.binary' as a pickle 

I'm at a bit of a loss here, so if anybody could help that would be much appreciated.

Thanks! 

[Jan C Peters]

I am not sure about the transformer preprocessing (since I did never use caffe's facilities for preprocessing), but what I do know is that you need a "deploy" type of network prototxt for this endeavor, i.e. one without data layers but instead a spec of its input blobs (defined by input and input_shape properties). If you put the same sames for these as for the top blobs of the data layers of the original network you can use the remaining prototxt content as-is. If you don't do this, the "net.blobs['data'].data[...] = ..." call will have no effect, because the input layers will load data into the "data" blob overwriting what you put into it when net.forward() is called.

Jan

[Yi Ji]

net.forward(start='xxx',end='xxx')