How to mark the area with bounding box after classification in caffe

[Guru Raj]

Hi Masters,

I am new to caffe. I was able to successfully test caffe on imagenet and other examples. I have also trained my own set of images using the imagenet support files (solver.prototxt/traib_val.prototxt etc).

Now after training the model, I am able to test and the model gives back some probability after classifying it as per synet_words.txt. I can see the probability on the console.

My question is how can I mark the test image (say cat.jpg) with a bounding box around the cat, once it is classified. i.e the test image cat.jpg has to have a bounding box around the objects and that should be the output. Also my question is can, I mark multiple objects in the same image (say cat and car in the cat.jpg, if the cat is sitting on a car). I am not able to figure out how to do this.

Please guide me.

Thanks in advance.

PS: I have interfaced caffe with python and not matlab.

Best Regards,

Guru

[Param Rajpura]

Hi Guru Raj,

Your answer lies in the caffe-users group...you may seach for sliding window detector and localization problem.....for detailed and informative answers...

To start with, this tutorial will do the job for you

https://github.com/BVLC/caffe/blob/master/examples/detection.ipynb

Happy Brewing!
Param Rajpura

[Guru Raj]

Hi Param,

Thanks for the quick reply and the link to the guide.

I see the following para in the example.

First, we'll need region proposals and the Caffe R-CNN ImageNet model:

• Selective Search is the region proposer used by R-CNN. The selective_search_ijcv_with_python Python module takes care of extracting proposals through the selective search MATLAB implementation. To install it, download the module and name its directory selective_search_ijcv_with_python, run the demo in MATLAB to compile the necessary functions, then add it to your PYTHONPATH for importing. (If you have your own region proposals prepared, or would rather not bother with this step, detect.py accepts a list of images and bounding boxes as CSV.)

!echo `pwd`/images/fish-bike.jpg > _temp/det_input.txt

Questions:

1. I dont want to use matlab. In that case, how do i generate my own proposal. i.e how to i generate the co-ordinates which are fed as input to detect.py ?

2.  What sould the det_input.txt contain? 

Thanks, i am almost there...if you can clear these answers I would be greatful to you.

Best Regards,

Guru

[Param Rajpura]

Hi Guru,

As per the understanding i have of R-CNN, the selective search algorithm is a smarter way of finding the regions where probability of object being there is high....

You too go through that paper its really helpful if you want to make your object localization work better...

So,

 1. if you dont want matlab.....

a. refer the paper and you might have to implement the algo..

b. use the general sliding window approach(to start with keep that fixed size same as your input blob size) and make predictions on each window....draw a bounding box with the highest prob

2. det_input.txt contains the path to the input file (image) on which you want to locate the objects....in your case cat.jpg

Hope that helps!