How to fine-tune a model using images that have bounding boxes?

[Md. Atiqur Rahman]

Hi all,

Could you please let me know if I can fine-tune a caffe model using images that have specified bounding box information with one bounding box per image?

[Yasser Souri]

Salam

I had this similar setup, what I did was to first crop each image to the bounding box using a simple script and save those pre-cropped files to disk. Then I changed my train/test.txt files to use them instead of the full images. The rest is similar to the flickr style fine tuning example.

[Md. Atiqur Rahman]

Dear Brother Yaseer

walaikum assalam. I highly appreciate such a nice work around you provided. But, is there no other way in matlab wrapper to provide bounding-box info?

I have a few more questions. Since I am very new to Caffe, these might sound very silly to you.

1) After I get the fine-tuned model, how can I extract features using this model in matlab?

2) My data set has got only training images and testing images. But, for fine-tuning I see I need to provide a train.txt and test.txt file listing the images files. I guess, these two files have to be generated from my training image set only by splitting it into train.txt and test.txt. And, my original testing image set would be used for final testing. Please correct me if I am wrong.

[Md. Atiqur Rahman]

Dear Brother @Yaseer,

It would be of great help to me if you kindly response to my below queries.

[Yasser Souri]

No I meant before using the caffe's matlab wrapper you have to pre-crop your images and then treat them as regular images afterwards.

1. you can do a forward pass, then use the fc6 or fc7's layer values as features.

2. you can split your training data into train/val sets as you like. But still giving the actual test set while fine-tuning should not be a problem after you have decided the network parameters.

I hope this helps

[Md. Atiqur Rahman]

Thanks a lot Brother.

1. How to do a forward pass using matlab wrapper in Caffe? I see only to find out the scores using matlab. I would highly appreciate if you could let me know the steps to find out the features using matlab wrapper.

Regards
Md. Atiqur Rahman

[Yasser Souri]

Sorry but I don't use Matlab. Try looking for Matlab examples in caffe's website.

[Md. Atiqur Rahman]

Thanks a lot Brother. I didn't find any example for extracting features with matlab.

Then how do you usually extract features using Caffe? Using this script (https://github.com/BVLC/caffe/tree/master/examples/feature_extraction)? But, it returns a levelDB file and I cannot convert it to .mat file.

[Yasser Souri]

I meant using this example: http://nbviewer.ipython.org/github/BVLC/caffe/blob/master/examples/filter_visualization.ipynb

you can use the fc7 or fc6 layer filters which are 4096 dimension vector as features.

[Md. Atiqur Rahman]

I would like to highly appreciate your cordial help. Thanks a lot brother.

[Md. Atiqur Rahman]

Dear Brother @Yaseer,

Assalamualaikum. Now that I have fine-tuned my model based on the example of flickr fine tuning using a total of 3999 images (with 2800 images for training and the rest 1999 for validation), I find that the accuracy after 100,000 iterations is only 0.016 as shown below.

Is it OK? Or, I have got something wrong in the fine tuning process.

I would highly appreciate your as usual cordial help in this regard.

.....
I1230 16:43:29.236649 28028 solver.cpp:228] Iteration 100000, loss = 3.71897
I1230 16:43:29.236671 28028 solver.cpp:247] Iteration 100000, Testing net (#0)
I1230 16:43:56.586487 28028 solver.cpp:298]     Test net output #0: accuracy = 0.016
I1230 16:43:56.586660 28028 solver.cpp:233] Optimization Done.
I1230 16:43:56.586663 28028 caffe.cpp:121] Optimization Done.

[Yasser Souri]

It is not okay I guess. I tried with the same parameters as the flickr finetune example and everything worked out. But I don't know what might be your problem.

[Md. Atiqur Rahman]

Thanks a lot Brother....I am now doing bit-to-bit copy of the Flickr example to see if really works in my system. If it is OK, then I will repeat the process with my dataset.

By the way, while extracting features using the python script or the linux shell script as provided with Caffe, it asks for a mean file which is by default the ImageNet mean file. For extracting my dataset's feature on ImageNet pretrained model or ImageNet fine-tuned model, do I need to create my dataset's mean file? Or, the ImageNet mean file is OK?

[Yasser Souri]

I think going with the imagenet mean file would be ok.

[Md. Atiqur Rahman]

Thanks a lot Brother.... I would like to highly appreciate your so kind and cordial feedback and wish to receive the same in future, if required.

[Md. Atiqur Rahman]

Dear Brother Yasser,

Assalamualaikum.... I have tried the the python code (http://nbviewer.ipython.org/github/BVLC/caffe/blob/master/examples/filter_visualization.ipynb) to extract features. But, while executing the command

import caffe

it produces the following error -

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "../python/caffe/__init__.py", line 1, in <module>
    from .pycaffe import Net, SGDSolver
  File "../python/caffe/pycaffe.py", line 11, in <module>
    import caffe.io
  File "../python/caffe/io.py", line 6, in <module>
    from caffe.proto import caffe_pb2
  File "../python/caffe/proto/caffe_pb2.py", line 3, in <module>
    from google.protobuf import descriptor
ImportError: No module named google.protobuf



I do not know the reason behind. Could you please advise?