Several questions related to image classification after training LeNet with my own database

[mprl]

Hello,

I'm currently trying to classify images with LeNet after training it with my own database (9000 images for training, 2000 images for testing, digits (0 to 9), grayscale, 50x100). I converted all the images to two lmdb database (train and test) with the convert_imageset.

After the training, i got an accuracy of 99%. But when i try to test single images manually with classification.bin, it goes wrong 90% of the time.
For exemple, a pretty clear image of a 5 is predicted at 100% as a 4.

---------- Prediction for ../DB/DBtest/image_test.bmp ----------
1.0000 - "Four"
0.0000 - "One"
0.0000 - "Three"
0.0000 - "Zero"
0.0000 - "Two"

Here come the questions :

1) With the LeNet_train_test.prototxt, the input data (train and test) is scaled with a factor of 0.00390625 in order to change the range from [0;255] to [0;1]. Have i to scale my .bmp image like this when i test them with classification.bin ?
My first layer in lenet_deploy.prototxt is an Input type layer. And i can't find a way to apply the scale factor with this layer (transform_param does not exist with Input type layer). Have i to manually rescale every single .bmp i want to test between 0 and 1 ? If yes, how ?

2) classification.bin need a txt file with all label possible. So i wrote a txt file with a label on each line. But i wonder how can he know which output neuron is asociated with which label. Is the first output neuron associated with 0 and the last to 9 ? There is apparently no reason to that during the learning process. How can i know which output give the probability of which label after the last softmax layer ?

3) I also generated a mean_file of my database (the training one, not the testing one) with the compute_image_mean tool of Caffe. But i wonder why is this needed for classification.bin, and what mean is it exactly. The mean has to be a number between 0 and 255 or between 0 and 1 ? Did i need to rescale this too ? And if yes, how ?

Thanks,

[Jan]

1. Yes, of course you need to apply the same preprocessing to the images as in training. I am not sure about classification.bin, but I suspect it does not apply preprocessing. Well, you can have a look in the code to clarify.

2. Usually zero-based integers are used to refer to classes, as it is easier to handle than string labels. So if you have ten classes, they will be identified with the labels 0,1,..,9. This is the way the accuracy and softmaxwithloss layers require the labels to look like. Whether 0 actually means "bird" or "the digit zero" is of no concern to caffe. That in your case class 0 actually means 'the digit zero' is just pure coincidence. With that in mind, it just makes sense to associate the first output neuron with class 0, the second with class 1 and so on. So all in all, caffe does not need to know what the numeric labels mean, but you have to!

3. Well, its an "average image", computed over all images in the database. It has the same shape as every other image, i.e. same number of channels, width and height. Only that every pixel value is the mean of all the pixel values at that same position in all the average images.

If you used mean subtraction in training (which is a form of preprocessing) then you have to subtract the mean during inference too!

Jan

[mprl]

Ok, so if i trained my network with that input layer :

name: "LeNet1"
layer {
  name: "mnist"
  type: "Data"
  top: "data"
  top: "label"
  include {
    phase: TRAIN
  }
  transform_param {
    scale: 0.00390625
  }
  data_param {
    source: "examples/train_lmdb"
    batch_size: 64
    backend: LMDB
  }
}

I have to apply scale of 0.00390625 to every pixel of my .bmp image (any tool to do it ?) before testing it with classification.bin and i dont have to substract the mean.
I seems that i can't tell to calssification.bin that i don't want to substact the mean.
How can i substract the mean before training ? I can't find any option to do that with a Data type layer.

[Jan]

Actually it seems from your prototxt snippet that you don't subtract the mean.

Subtracting the mean works through the transformer (transform_param), see https://github.com/BVLC/caffe/blob/master/src/caffe/proto/caffe.proto#L419.

usually it's best to write your own script for inference, as there usually is no one-fits-all. The classification.bin tool also was intended to show how to use caffe's C++ API, not be able to do any kind of classification/inference.

Jan