Hello Caffe users! I have a philosophical question.
I'm fine-tuning the provided 'network-in-network' model for a separate dataset of images of my own choosing. I computed the mean image of this new dataset and changed the "data" layer to match the new dataset *and* its mean file.
When I used the Python bindings to get features out (and thus get the nice 10-crops-averaging/ensemble) behavior, I forgot to subtract the proper mean. I accidentally subtracted the Imagenet mean because that was the default parameter. It worked well enough (~71% accuracy), but when I change to my own dataset's mean (what the network was fine-tuned with), performance improved to about 73%. Incidentally, this is a few % higher than the output of Caffe's testing interval, likely because of the cropping/ensembling behavior.
My question is: Should I have changed the image mean when fine-tuning/training the network in the first place? Since "most" of the model was trained on imagenet mean and not my own dataset's mean, perhaps it could give better performance.
Does anyone have some "in-the-trenches" suggestions they can share?