Confusion with caffe.TEST

[Kağan İncetan]

Hi,

I have trained a model and now I want to see the prediction for a single image.

To do that, I have created deploy.prototxt and I have weights from the trained model.

I got confused when I define the Net because it has net = caffe.Net(model, weights, caffe.TEST)

So if I want to use a single image which I have not defined as test data before then what does caffe.TEST refer to ?

Please excuse me as I am a new user.

Regards

Kagan

[Przemek D]

I encourage you to check the examples first, particularly the 00-classification notebook.

[Kağan İncetan]

Thank you for the reply.

I understand that caffe.TEST is used to state we are defining net for testing.

Now I need further help for deploying because I am doing regression instead of classification. Hence I am not sure if I can use these examples directly.

I got confused with mean image because I computed image means for train and test set before I started the training session. But now I want to use only one image do I still need to conver it to LMDB and then compute image mean?

Thanks in advance

5 Ekim 2017 Perşembe 08:28:25 UTC+2 tarihinde Przemek D yazdı:

[Alex Ter-Sarkisov]

Some layers, e.g. batch normalization work in a different way in the train and test (hence you have to specify that). I think what you mean is that you need two different config files, deploy.prototxt instead of train_val.prototxt. In deploy.prototxt the input is usually defined in a different way, e.g. as an image rather than lmdb dataset, and the loss layer is removed + certain other changes (e.g. crop parameters, if the image size is different) 

[Kağan İncetan]

Hi Alex,

Thank you for the reply.

I am getting better understanding. As you stated, I have deploy.prototxt which is modified version of train_prototxt and I get that it does not need lmdb type. So I am providing .png format and reading my last layer to get the prediction which is the position in XYZ coordinates for my case.
However, even though the result of test sessions were okeyish the result of deploy of an image that I have not provided model before has a huge error.

I am asking, before I trained the model I divided the set as train and test datasets and I gave the test data as well in advance. The result of this dataset was fine but when I give another image upon deploying I am getting huge error. Does that make sense? Because that image that I am using upon deploying is quite similar to the ones from train or test dataset. (as I have collected images in a small area so that image may even be almost same with one of from train or test dataset)

P.S. I realized that I may be using test and validate datasets interchangbly

Regards
5 Ekim 2017 Perşembe 15:04:58 UTC+2 tarihinde Alex Ter-Sarkisov yazdı:

[Alex Ter-Sarkisov]

You could be simply overfitting to the train data. Could you attach or leave a link to your train_val and deploy prototxt files, I'll have a look.

[Kağan İncetan]

Please see them as attached. Thank you very much for your help.

The data I have is images that I took with a camera and labels are the positions in X Y Z and quaternions for orientation which I have only in one axis.

Looking forward to hear you.

regards3

5 Ekim 2017 Perşembe 17:56:49 UTC+2 tarihinde Alex Ter-Sarkisov yazdı:

[Alex Ter-Sarkisov]

Nothing seems to be screamingly wrong, but a few things are a bit baffling/unusual: 

1) In train_val.txt you use a slice layer for labels, I've never used it myself, so not sure how it works

2) You use crop_size=224, same as in googlenet. Does that really match your input image size? 

3) You use Euclidean loss instead of Softmax w/Cross-entropy. Euclidean may produce small derivatives. Did you check net.blobs[blob_name].diff? If they are all 0s, your network simple doesn't learn anything. 

4) The input in deploy.txt is an image size 224x224, same as crop size in the training data. Are your images of different size in training/deploy phases? 

5) Finally, your output is a score over 4 classes without softmax normalization as in GoogleNet.  

Hope it helps. 

[Kağan İncetan]

Dear Alex,

Thank you very much again for your time.


I find good to mention that I am using the following example
http://arxiv.org/abs/1505.07427 Alex Kendall, Matthew Grimes and Roberto Cipolla "PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization." Proceedings of the International Conference on Computer Vision (ICCV), 2015.

For your questions

1) As I am a new user I also have no much info but I am using the same size of images with quaternion positions which is exactly same inputs as the Alex Kendal example. Hence I dont think there will be problem.
2) Yes, because I am using the same size of images.
3) I have checked net.blobs[blob_name].diff. I have 2 Euclidean loss layers and outputs are 1 for loss_xyz and 500 for loss_wpqr
4) I also provide same size of images upon deploying
5) I couldnt get this

I have trained two models by using same layers. In the first one I have used gray scale images and I got validation error which is unexptectedly high.
Then I collected RGB color images by using different camera and the results were almost same even though the color images were slightly better.

I am planning to train one more time by using higher amount of data this time expecting to get better results. But I still dont get deploy part because it gives me totally bad results than validation results.

I am confused with the following. What does exactly validation results represent ? As I say to my model I am gonna use these images as validation before I start my training, then is it normal that I am getting better results with validation data then an image that model has not seen before ?

Can you check my script that I am using to get deploy results? I doubt that I may be trying to output incorrect data.
Also. I am providing the script that I am using to get validation results which I got from Alex Kendal's example.

I am looking forward to hear you

Regards

6 Ekim 2017 Cuma 13:02:52 UTC+2 tarihinde Alex Ter-Sarkisov yazdı:

[Alex Ter-Sarkisov]

'2 Euclidean loss layers and outputs are 1 for loss_xyz and 500 for loss_wpqr' Are 1 and 500 derivative values? That's very strange, it's usually something very small. 

Did you check how you made you lmdb files btw? It's a good idea to permute the inputs while keeping observation:label consistency, because lmdb reads data from the first to the last entry simulatenaously from data and label files. 

Using transformer is not a good idea. Also I noticed you don't subtract the means. I do it like this: 

import numpy as np

import os,sys

#sys.path.insert(0,'/home/ICTDOMAIN/453615/Downloads/caffe-crfrnn/python')

sys.path.insert(0,'/home/ICTDOMAIN/453615/Downloads/caffe/python')

import caffe

import pickle

import matplotlib.pyplot as plt

import cv2

from PIL import Image as PILImage

caffe.set_mode_gpu()

caffe.set_device(0)

im='1000.png'

frame_num=im

fcn_dir = "/home/fcn.berkeleyvision.org/voc-fcn8s/"

MODEL_FILE = fcn_dir + 'deploy.prototxt'

PRETRAINED = fcn_dir + 'fcn8s-heavy-pascal.caffemodel'

net = caffe.Net(MODEL_FILE, PRETRAINED, caffe.TEST)

mean_vec = np.array([103.939, 116.779, 123.68], dtype=np.float64)

image = caffe.io.load_image(im)

image = 255 * image[:,:,::-1]

image -= mean_vec

cur_h, cur_w, cur_c = image.shape

caffe_in = np.zeros((1, cur_c, cur_h, cur_w), dtype=np.float64)

caffe_in[0] = image.transpose((2, 0, 1))

net.blobs['data'].reshape(*caffe_in.shape)

net.forward()

Hope it helps.

Alex

[Kağan İncetan]

Hi Alex,

Thanks for the reply.
 
Could you please tell me how you compute image mean without lmdb? Because I create lmdb files by using the attached document which takes labels(positions for my case) and images as you mentioned but for deploying I assume that I dont need labels as input.

Also, I have printed net.blobs[loss3/loss3_xyz].diff net.blobs[loss3/loss3_wpqr].diff and gave me 1 and 500 but they are also loss weights so not sure if they are derivatives

9 Ekim 2017 Pazartesi 17:01:59 UTC+2 tarihinde Alex Ter-Sarkisov yazdı:

[Alex Ter-Sarkisov]

Yeah because dE/dE = 1, not sure where 500 comes from. You should look at derivatives for lower/deeper layers, e.g.

net.blobs['icp9_out0'].diff

. If they are 0s or close it means that the network doesn't learn anything. 

The script I wrote above can be used without lmdb - just subtract the means before input into the network, Just make sure you do it before converting RGB to BGR. Same with LMDB - subtract the means etc before you put he file into LMDB dataset. There are plenty of tutorials on how to do it. All I said you should make sure the order of data/labels is the same , e.g. {data1, label1}, {data2, label2}, etc. 

Alex 

[Kağan İncetan]

Hi Alex,

I have computed image mean with the following
img = cv2.imread(im)
mean_vec = np.average(np.average(img, axis=0), axis=0)

Then I am applying your code but I get Check failed: K_ == new_K (2048 vs. 49920) Input size incompatible with inner product parameters.
error. Do you have any idea ?

Regards

9 Ekim 2017 Pazartesi 19:30:18 UTC+2 tarihinde Alex Ter-Sarkisov yazdı:

[Kağan İncetan]

I realized that I needed to resize the image as I was doing for training. So now the error is solved.

However, @Alex with your deploy code I am getting the same predictions for different images. I also doubt finding where exaclty do you say your model to find the prediction for given image?

Thanks for the help in advance

Regards

10 Ekim 2017 Salı 12:08:38 UTC+2 tarihinde Kağan İncetan yazdı: