Impact of input_scale in classification results

[Vincent]

Hi,

I have detected an impact of the classification parameter "input_scale" on results. 

To precise the context, I have followed this standard procedure for the training:

- creation of train.txt and val.txt (I have RGB images; TRAIN : 470 images for class 0 and 64 for class 1 ; VAL : 118 for class 0 and 16 for class 1)

- use of create_imagenet.sh pour resize (256*256) and create train_levelDB and val_levelBD

- use of make_imagenet_mean.sh to create imagenet_train_mean.binaryproto and imagenet_val_mean.binaryproto

- use of models/bvlc_reference_caffenet/train_val.prototxt (modification of source, mean_file and num_output).

- use of models/bvlc_reference_caffenet/solver.prototxt (change for using 250 iterations max because my learning set is small)

- training: ./build/tools/caffe train --solver=models/bvlc_reference_caffenet/solver.prototxt :

after 5h, the result seems good :

I0528 05:18:33.718219 22302 solver.cpp:248] Iteration 250, loss = 0.174455

I0528 05:18:33.718261 22302 solver.cpp:266] Iteration 250, Testing net (#0)

I0528 05:19:21.391991 22302 solver.cpp:315]     Test net output #0: accuracy = 0.892

I0528 05:19:21.392213 22302 solver.cpp:315]     Test net output #1: loss = 0.236802 (* 1 = 0.236802 loss)

I0528 05:19:21.392251 22302 solver.cpp:253] Optimization Done

To test the model, I try to classify images of the VAL dataset.

net = caffe.Classifier(MODEL_FILE, PRETRAINED, mean=np.array([numpy.mean(arr[0][0]),numpy.mean(arr[0][1]),numpy.mean(arr[0][2])]), channel_swap=(2,1,0), raw_scale=255, image_dims=(256, 256), input_scale=1)

Result : 98% of correct classification for class0 ; only 43% for class1

It is not really close to expected results (Test net output #0: accuracy = 0.892; loss = 0.236802), isn't it ?

I have tested different values for parameters, and input_scale has an interesting influence, for instance with :

 - input_scale=5 ==>  85% of correct classification for class0 ; 71% for class1

 - input_scale=10 ==>  77% of correct classification for class0 ; 86% for class1

 - input_scale=12 ==>  75% of correct classification for class0 ; 86% for class1

Here I am closer to tests performed in the training step.

Do you know how to explain that ?

Thanks, 

Vincent