follow the MNIST demo .But actual "detect" accuracy not matching validation accuracy .

[8748...@qq.com]

Hi, everyone.
I want to use LeNet7 to do classification for three classes remote sensing pictures: ridge, unknown and valley.
So I prepare the trainDataSet(ridge, unknown and valley) and the testDataSet(ridge, unknown and valley).
I follow the MNIST demo .But actual "detect" accuracy not matching validation accuracy .

Train set:
(ridge: 15000 32*32*1 images, unknown : 15000 32*32*1 images, valley: 15000 32*32*1 images)
Test set:
(ridge: 630 32*32*1 images, unknown : 775 32*32*1 images, valley: 704 32*32*1 images)

Steps:
First, I dscompile these data using the command:
dscompile E:\0download_wh\20140903New\train -outdir E:\0download_wh\20140903New\eb_datasetTrain -dname mnist_train -dims 32x32x1 -kernelsz 7x7
dscompile E:\0download_wh\20140903New\test -outdir E:\0download_wh\20140903New\eb_datasetTest -dname mnist_test -dims 32x32x1 -kernelsz 7x7

Second, I train the dscompiled data to get the weight file. After a few iterations, the test_correct can reach 97%.

Testing on 2109 samples...i=11 name=train [495001] sz=45000 energy=0.0649164 (class-normalized) errors=1.90222% uerrors=1.90222% rejects=0% (class-normalized) correct=98.0978% ucorrect=98.0978%
errors per class: 0_samples=15000 0_errors=2.04% 1_samples=15000 1_errors=1.85333% 2_samples=15000 2_errors=1.81333%
success per class: 0_samples=15000 0_success=97.96% 1_samples=15000 1_success=98.1467% 2_samples=15000 2_success=98.1867%

i=11 name=val [495001] sz=2109 test_energy=0.0808647 (class-normalized) test_errors=2.31986% test_uerrors=2.32338% test_rejects=0% (class-normalized) test_correct=97.6801% test_ucorrect=97.6766%
errors per class: test_0_samples=630 test_0_errors=2.22222% test_1_samples=775 test_1_errors=2.32258% test_2_samples=704 test_2_errors=2.41477%
success per class: test_0_samples=630 test_0_success=97.7778% test_1_samples=775 test_1_success=97.6774% test_2_samples=704 test_2_success=97.5852%

Third, I detect the testDataSet using the weight file in the second step, the results are different from the training stage.
accuracy per class: test_ridge_samples=630 test_ridge_success=6.35%
test_unknown_samples=714 test_unknown_success=85.4%
test_valley_samples=704 test_valley_success=0.568%

In my .conf file:
#arch = ${pp},${arch_${run_type}} # run_type is set by runned tool
arch = ${arch_${run_type}} # run_type is set by runned tool
weights = mnist_net00011.mat #my weight file

I am puzzled and do not know how to handle it. I hope the writer can help me. Thank you!

[8748...@qq.com]

(1)environment : 64-bit Installer eblearn-1.2_x64-r2631-win64.exe


(2)my .conf file
# paths ########################################################################
name = mnist
HOME = D:/WuHui #add by wuhui
ebl = ${HOME}/eblearn # eblearn root
#root = ${HOME}/mnist/ # mnist data root
#root = ${HOME}/eblearn/my_mnist # mnist data root, change by wuhui
root = ${HOME}/eblearn/20140903New/012/Data
tblroot = ${ebl}/tools/data/tables # location of table files

# network high level switches ##################################################
classifier_type = cf # type of classifier, cf: 2-layer, c: 1-layer
classifier_hidden = 16 # number of hidden units in 2-layer classifier
features_type = cscs # type of features
nonlin = tanh # type of non-linearity
manual_load = 0 # manually load weights for individual modules
norm = 1 # use normalization layers
pool = subs # l2pool (is another option)
run_type = train # detect fprop

# features and classifiers #####################################################

#arch = ${pp},${arch_${run_type}} # run_type is set by runned tool
arch = ${arch_${run_type}} # run_type is set by runned tool

arch_train = ${features},${classifier}
arch_detect = ${arch_train}
arch_fprop = ${features}
arch_name = ${arch_name_${run_type}}
arch_name_fprop = ${features_name}
arch_name_train = ${features},${classifier}
features = ${features_${features_type}}
features_name = ${features_type}
classifier = ${classifier_${classifier_type}}

# energies & answers ###########################################################
trainer = trainable_module1 # the trainer module
trainable_module1_energy = l2_energy # type of energy
answer = class_answer # how to infer answers from network raw outputs

# normalization ################################################################
norm0_0 = # no normalization
norm2_0 = # no normalization
norm0_1 = wstd0 # contrast normalization
norm2_1 = wstd2 # contrast normalization

# architecture #################################################################

# features
features_cscs = ${c0},${s1},${c2},${s3}

# classifiers
classifier_c = ${c5}
classifier_cf = ${c5},${f7}

# main branch layers
c0 = conv0,addc0,${nonlin},abs0,${norm0_${norm}}
s1 = ${pool}1,addc1,${nonlin}
c2 = conv2,addc2,${nonlin},abs2,${norm2_${norm}}
s3 = ${pool}3,addc3,${nonlin}
c5 = conv5,addc5,${nonlin}
f6 = linear6,addc6,${nonlin}
f7 = linear7,addc7,${nonlin}

# main branch parameters
zpad0_dims = 5x5 # padding for this kernel size
conv0_kernel = 5x5 # convolution kernel sizes (hxw)
conv0_stride = 1x1 # convolution strides (hxw)
conv0_table = # convolution table (optional)
conv0_table_in = 1 # conv input max, used if table file not defined
conv0_table_out = 6 # features max, used if table file not defined
conv0_weights = # manual loading of weights (optional)
addc0_weights = # manual loading of weights (optional)
wstd0_kernel = ${conv0_kernel} # normalization kernel sizes (hxw)
subs1_kernel = 2x2 # subsampling kernel sizes (hxw)
subs1_stride = ${subs1_kernel} # subsampling strides (hxw)
l2pool1_kernel = 2x2 # subsampling kernel sizes (hxw)
l2pool1_stride = ${l2pool1_kernel} # subsampling strides (hxw)
addc1_weights = # manual loading of weights (optional)
conv2_kernel = 5x5 # convolution kernel sizes (hxw)
conv2_stride = 1x1 # convolution strides (hxw)
conv2_table = ${tblroot}/table_6_16_connect_60.mat # conv table (optional)
conv2_table_in = thickness # use current thickness as max table input
conv2_table_out = ${table1_max} # features max, used if table file not defined
conv2_weights = # manual loading of weights (optional)
addc2_weights = # manual loading of weights (optional)
wstd2_kernel = ${conv2_kernel} # normalization kernel sizes (hxw)
subs3_kernel = 2x2 # subsampling kernel sizes (hxw)
subs3_kernel = 2x2 # subsampling kernel sizes (hxw)
subs3_stride = ${subs3_kernel} # subsampling strides (hxw)
l2pool3_kernel = 2x2 # l2poolampling kernel sizes (hxw)
l2pool3_stride = ${l2pool3_kernel} # subsampling strides (hxw)
addc3_weights = # manual loading of weights (optional)
linear5_in = ${linear5_in_${net}} # linear module input features size
linear5_out = noutputs # use number of classes as max table output
linear6_in = thickness # linear module input features size
linear6_out = ${classifier_hidden}
linear7_in = thickness # use current thickness
linear7_out = noutputs # use number of classes as max table output
conv5_kernel = 5x5 # convolution kernel sizes (hxw)
conv5_stride = 1x1 # convolution strides (hxw)
conv5_table_in = thickness # use current thickness as max table input
conv5_table_out = 120 # features max, used if table file not defined

# preprocessing ################################################################
pp = ${pp_${run_type}}
pp_train = zpad0
pp_fprop = zpad0
pp_detect = zpad0
#pp_detect = resizepp0
#rgb_to_ypuv0_norm = 0
#threshold = .99 # threshold of [0,1] confidence

# possible preprocessings
pp_y = rgb_to_y0
pp_yuv = rgb_to_yuv0
pp_yp = rgb_to_yp0
pp_ypuv = rgb_to_ypuv0
rgb_to_ypuv0_kernel = 7x7
resizepp0_pp = rgb_to_ypuv0
resizepp0_zpad = 2x2x2x2

# training #####################################################################
classification = 1 # load datasets in classification mode, regression otherwise

#train = ${root}/train-images-idx3-ubyte # training data
#train_labels = ${root}/train-labels-idx1-ubyte # training labels
#train_size = 45000 # limit number of samples
#val = ${root}/t10k-images-idx3-ubyte # validation data
#val_labels = ${root}/t10k-labels-idx1-ubyte # validation labels
#val_size = 2100 # limit number of samples

train = ${root}/mnist_train_data.mat # training data
train_labels = ${root}/mnist_train_labels.mat # training labels
train_size = 45000 # limit number of samples
val = ${root}/mnist_test_data.mat # validation data
val_labels = ${root}/mnist_test_labels.mat # validation labels
val_size = 2109 # limit number of samples
#data_bias = -128 # bias applied before coeff
data_coeff = .01 # coeff applied after bias
add_features_dimension = 1 # samples are 28x28, make them 1x28x28
test_display_modulo = 0 # modulo at which to display test results

# hyper-parameters
##########
hessian_period = 7500
#########
eta = .0001 # learning rate
reg = 0 # regularization
reg_l1 = ${reg} # L1 regularization
reg_l2 = ${reg} # L2 regularization
reg_time = 0 # time (in samples) after which to start regularizing
inertia = 0.0 # gradient inertia
anneal_value = 0.0 # learning rate decay value
anneal_period = 0 # period (in samples) at which to decay learning rate
gradient_threshold = 0.0
iterations = 2 # number of training iterations
ndiaghessian = 100 # number of sample for 2nd derivatives estimation
epoch_mode = 1 # 0: fixed number 1: show all at least once
#epoch_size = 4000 # number of training samples per epoch. comment to ignore.
epoch_show_modulo = 4000 # print message every n training samples
sample_probabilities = 0 # use probabilities to pick samples
hardest_focus = 1 # 0: focus on easiest samples 1: focus on hardest ones
ignore_correct = 0 # If 1, do not train on correctly classified samples
min_sample_weight = 0 # minimum probability of each sample
per_class_norm = 1 # normalize probabiliy by class (1) or globally (0)
shuffle_passes = 1 # shuffle samples between passes
balanced_training = 1 # show each class the same amount of samples or not
random_class_order = 0 # class order is randomized or not when balanced
no_training_test = 0 # do not test on training set if 1
no_testing_test = 0 # do not test on testing set if 1
max_testing = 0 # limit testing to this number of samples,原来是0
save_pickings = 0 # save sample picking statistics
binary_target = 0 # use only 1 output, -1 is negative, +1 positive
#test_only = 0 # if 1, just test the data and return
#save_weights = 0 # if 0, do not save weights after each iteration
keep_outputs = 1 # keep all outputs in memory
training_precision = double #float
save_weights = 1 # if 0, do not save weights when training

# training display #############################################################
display = 1 # display results
show_conf = 1 # show configuration variables or not
show_train = 1 # enable/disable all training display.原来为0。
show_train_ninternals = 1 # number of internal examples to display.原来为1。
show_train_errors = 0 # show worst errors on training set
show_train_correct = 0 # show worst corrects on training set
show_val_errors = 1 # show worst errors on validation set
show_val_correct = 1 # show worst corrects on validation set
show_hsample = 5 # number of samples to show on height axis。5
show_wsample = 18 # number of samples to show on height axis。18
show_wait_user = 1 # if 1, wait for user to close windows


# detection ####################################################################
#classes = ${ebl}/demos/mnist/mnist_classes.mat # names and # of classes
#classes = D:/WuHui/eblearn/20140903New/012/conf/mnist_train_classes.mat # names and # of classes
classes = D:/WuHui/eblearn/20140903New/012/conf/mnist_test_classes.mat

scaling_type = 4 # 4: no multi-scaling, just original 28x28 image
input_gain = ${data_coeff} # use same gain as in preprocessed training data

camera = directory # input images come from a directory
input_dir = E:/0download_wh/20140903New/test/0
#next_on_key = n # with focus on display window, press "n" to process next
weights = mnist_net00011.mat #train时参数：mnist_net00011.mat; detect:mnist_net00005.mat



test_only = 0 # if 1, just test the data and return

################################################################################
#bbox_decision = 0
#nms = 0
bbox_show_conf = 1 # display confidence in each bbox or not
bbox_show_class = 1 # display class name in each bbox or not

[8748...@qq.com]

In the train and detect stage, the arch is the same. Both don't use zpad0.

What's the difference between zpad0 and resizepp0_pp. And what's them for?