CNN Feature Extraction produces random classification with SVM

[thecro...@gmail.com]

Hi all.

I'm trying to solve a fine-grained classification problem using Caffe. My first attempt is to extract CNN features from fully connected layers and train a linear SVM on top of the extracted features. I'm using the Caffenet Reference implementation, initially.
Literature says that fc6 or fc7 should contain good features to datasets others than Imagenet, such as Caltech-UCSD Birds 200.

The only problem is that the trained SVM produces predictions with accuracy pretty randomic (~22% with 5 classes).

I strongly suspect there is some mistake in my approach or in the code. Please, does someone have an hint?

Here are the steps I'm following (code is Python)

from sklearn.preprocessing import normalize

from sklearn.svm import LinearSVC

## checked if the data layer in train_val.prototxt contains the right path

net = caffe.Net('train_val.prototxt', 'reference_pretrained_model.caffemodel', caffe.TEST)

## for all training and testing data
net.forward()
features = net.blobs['fc7'].data
##

train_data = normalize(train_features)
test_data = normalize(test_features)
svc = LinearSVC(C=1.0)
svc.fit(train_data, train_labels)
svc.predict(test_data)

 

[Uni A]

Hi. Did you solve this problem?

[thecro...@gmail.com]

Yes, it turned out to be a bug in my code.

I was modifying the 'train_val.prototxt' file  AFTER  the instantiation of the caffe.Net object. Instead, the protobuf structure is actually loaded in memory when instantiating the caffe.Net object.

Another problem that costs me some headaches was while extracting features, as in

features = net.blobs['fc7'].data

When extracting from several mini-batches, you need to take care of COPYING the actual data instead of overwrite the memory. So, the line above becomes

features = net.blobs['fc7'].data.copy()

I hope it helps!

[Komal Sinha]

Hi.

How did you extract CNN features? when I followed the tutorial demo to extract_feature in caffe (http://caffe.berkeleyvision.org/gathered/examples/feature_extraction.html) it results in data.mdb and lock.mdb files. Please tell me how to access these files for further classification by svm? or your approach to the problem.

Thanks !

[thecro...@gmail.com]

I extract features using Python interface. The code I posted above is pretty clear, yet not complete.

I never used the binary extract_features, but it seems to create LMDB files, which you can read using your favorite programming language.
I suggest you to study how to use the Python interface through the ipython notebooks in:

http://nbviewer.ipython.org/github/BVLC/caffe/tree/master/examples/

[mhaoy...@gmail.com]

first keep the model unchanged, train it, after trained well ,

set HingeLoss instead of SoftmaxWithLoss  in the loss layer model file.

and set learning rate to 0 in the conv layers and the last second FC layer, then retrain the net again, so the last layer is of SVM

 but I still have a question, how to set model text while classify images.

any body know it ?  take Hinge layer instead of Softmax layer ?

在 2015年9月11日星期五 UTC+8下午7:02:41，Komal Sinha写道：

[nali...@gmail.com]

Hi,I get the fc7 feature,how can i get train_features and test_features from it and train a linear SVM on it.can you give me  the complete code ?thx

在 2015年6月15日星期一 UTC+8下午9:07:24，thecro...@gmail.com写道：