test result with lmdb and .jpg files are different...

[gwang been park]

I doing CNN image classification with caffe.

I encountered very weird thing. If i test my trained network with test lmdb accuracy is 98%, but when I test my trained network with .jpg files (exactly same files with test lmdb) accuracy is 95%... (I use cpp_classification example in caffe library, i just edited it little and not core part.) I don't know why this happen. (and nvidia DIGITS result is same with lmdb result, only c++ jpg file classification result is different.)

I checked my c++ classification code's cv::resize is same with what caffe doing at convert_imageset.

And i have really no idea why this is happen..

please hlp me..

[Jan]

Are you sure you are using the same number of images for testing (see test_iter and batch_size parameters)? Also I'd try to make sure that what is fed into the network is really exactly the same in both cases. Maybe you can do some printf debugging if you want to go deeper?

Jan

[Hongxin Liu]

I am also deeply troubled by this problem these days.

I just saw a reply in https://groups.google.com/forum/#!searchin/caffe-users/cpp_classification/caffe-users/c06ZGV4s1TA/1CMncdqjDwAJ

Felix replied it: 

I'm the author of the cpp_classification code. Please note that this code is in the 'example' folder and not in the 'tools' folder. I contributed this example in the hope that it would be useful for advanced users that want to use the C++ API of Caffe in their application/library. It was meant to be a tutorial and not an universal classifier. 

Maybe there is something different between caffe test and cpp_classification. Check the code of them may help. I will do it soon. 

If you get some progress, please tell me, thank u :) 

Hongxin

在 2016年4月24日星期日 UTC+8上午3:45:48，gwang been park写道：

[gwang been park]

I found problem in pre-processing step. I used tools/convert_imageset.cpp for constructing my train and valid lmdb. And at my c++ classifier i did preprocessing manually. So i see the caffe/data_transformer.cpp code(that used by data layer). And change my manual preprocessing code same as data_transformer.cpp's one. But still i has 1% worse accuracy at manual classifier than caffe test. below is what i did at manual preprocessing step. my sample is gray image.

cv::Mat img = imread(xx.jpg, -1);

img.resize(30,30) (convert_imageset.cpp also use same cv::reisize function. option is also same.)

img.convertTo(img, CV_32FC1);

img = (img - mean_) * scale; (i used same scale parameter and mean.binaryproto with my train_val.prototxt datalayer's transform_param, and load mean code is same as example/cpp_classification) (caffe/data_transformer.cpp also do exactly same operation but there code is " for(for((img[] - mean_[])) * scale)) " but i think it is just same operation.

 img.copyTo(input blob)

forward.

so now, i think i did same operation with caffe test. But still now i get 1% worse result. So i want know why..... And i wish caffe will provide preprocessing layer for deploy.prototxt.... that can be same with train_val.prototxt.. This kind of problem is so tricky and hard to find reason.

2016년 4월 30일 토요일 오전 2시 40분 53초 UTC+9, Hongxin Liu 님의 말:

[fsr]

Maybe data shuffling?

23 Nisan 2016 Cumartesi 22:45:48 UTC+3 tarihinde gwang been park yazdı: