batch feature extraction with python

[Eder Santana]

Hi,

I checked the notebook for feature extraction online. 

My question is there any way to access the features of a list of input images in python?

Here is what happened when I tried:

I can see that net.predict accepts a list of images as input.

Indeed, the scores output report the classification result of all the input images. But what about the features?

For instance, I keep getting net.blobs['fc7'] with shape [10, 4096, 1, 1] independently of the size of the input image list.

Does 'net' keep the features of the input batch somewhere else?

Best,

Eder

[Eder Santana]

Just in case anybody else is interested, my provisory solution is attached. 

Nevertheless, I'm not marking this as complete because I'm hoping there is a cleaner method to do this.

Basically, my solution is a script that calls the 'extract_features.bin' several times. Note that I do that through system calls (I'm sorry!).

I did this because 'extract_features.bin' batches at most 500 images at a time, as I verified with experiments and as mentioned at another thread. Regardless of how many images we list at 'file_list.txt'. 

After each batch, the script load the features from the output leveldb

At the end it saves all the features as a numpy array. I think this should be faster than extracting features of one image at a time as showed at the feature extraction notebook. The ugliest part of it is that it should be loading the model to memory every time we call it...

But, while we wait for something better, here is a sample usage:

let your *.jpg images be at ./data:

$ python caffe_features.py ./data ./results [-d GPU|CPU] [-f jpg]

you can also import caffe_features at a python section and use the extract method F = caffe_features.extract('./data')

*Note that your edited 'imagenet_val.prototxt' should be on the same folder as caffe_features.py

**The only prereq (besides what we usually have) is the natsort module (it is easy to pip install, though).

[Mohamed Omran]

That's because the network processes the list of images in batches, the size of which is in this case set in ${CAFFE_ROOT}/examples/imagenet/imagenet_deploy.prototxt as follows:

name: "CaffeNet"

input: "data"

input_dim: 10 => input batch size (increase this to process more images at once)

input_dim: 3

input_dim: 227

input_dim: 227

[...]

This is also why the first dimension of a feature blob has size 10. The features you're accessing after the network is done predicting correspond to the last 10 input images(*) processed by the network. If you want to access more features, either increase the batch size, or modify e.g. predict() in python/caffe/classifier.py to store them before they're overwritten by the features of a subsequent batch.

(*)  If you look at the definition of predict in python/caffe/classifier.py, you'll see that the parameter 'oversample' is set to True by default. What this does is produce 10 crops from each image which each count as a separate element of an input batch. So technically with a batch size of 10 you're just processing a single image per forward pass. (Note that the predictions for crops of a single image are averaged.) This also needs to be taken into account if you want to find the correspondence between feature and input image.