Actual "detect" accuracy not matching validation accuracy

[essab...@gmail.com]

I get accuracy up to 99.9593% on the validation set as indicated in the command line. However, when I try the network on the validation set using "detect", the results are about 10%.

What could be the reason for that?

[8748...@qq.com]

在 2014年5月26日星期一UTC+8下午6时06分10秒，essab...@gmail.com写道：

> I get accuracy up to 99.9593% on the validation set as indicated in the command line. However, when I try the network on the validation set using "detect", the results are about 10%.
>
> What could be the reason for that?

I came across the same problem. Did you resolve it ?

I want to use LeNet7 to do classification for three classes remote sensing pictures: ridge, unknown and valley.
So I prepare the trainDataSet(ridge, unknown and valley) and the testDataSet(ridge, unknown and valley).
I follow the MNIST demo .But actual "detect" accuracy not matching validation accuracy .


Train set:
(ridge: 15000 32*32*1 images, unknown : 15000 32*32*1 images, valley: 15000 32*32*1 images)
Test set:
(ridge: 630 32*32*1 images, unknown : 775 32*32*1 images, valley: 704 32*32*1 images)


Steps:
First, I dscompile these data using the command:
dscompile E:\0download_wh\20140903New\train -outdir E:\0download_wh\20140903New\eb_datasetTrain -dname mnist_train -dims 32x32x1 -kernelsz 7x7
dscompile E:\0download_wh\20140903New\test -outdir E:\0download_wh\20140903New\eb_datasetTest -dname mnist_test -dims 32x32x1 -kernelsz 7x7



Second, I train the dscompiled data to get the weight file. After a few iterations, the test_correct can reach 97%.

Testing on 2109 samples...i=11 name=train [495001] sz=45000 energy=0.0649164 (class-normalized) errors=1.90222% uerrors=1.90222% rejects=0% (class-normalized) correct=98.0978% ucorrect=98.0978%
errors per class: 0_samples=15000 0_errors=2.04% 1_samples=15000 1_errors=1.85333% 2_samples=15000 2_errors=1.81333%
success per class: 0_samples=15000 0_success=97.96% 1_samples=15000 1_success=98.1467% 2_samples=15000 2_success=98.1867%

i=11 name=val [495001] sz=2109 test_energy=0.0808647 (class-normalized) test_errors=2.31986% test_uerrors=2.32338% test_rejects=0% (class-normalized) test_correct=97.6801% test_ucorrect=97.6766%
errors per class: test_0_samples=630 test_0_errors=2.22222% test_1_samples=775 test_1_errors=2.32258% test_2_samples=704 test_2_errors=2.41477%
success per class: test_0_samples=630 test_0_success=97.7778% test_1_samples=775 test_1_success=97.6774% test_2_samples=704 test_2_success=97.5852%


Third, I detect the testDataSet using the weight file in the second step, the results are different from the training stage.
accuracy per class: test_ridge_samples=630 test_ridge_success=6.35%
test_unknown_samples=714 test_unknown_success=85.4%
test_valley_samples=704 test_valley_success=0.568%

In my .conf file:
#arch = ${pp},${arch_${run_type}} # run_type is set by runned tool
arch = ${arch_${run_type}} # run_type is set by runned tool

weights = mnist_net00011.mat #my weight file

I am puzzled and do not know how to handle it. I hope the writer can help me. Thank you!

[essab...@gmail.com]

On Monday, May 26, 2014 2:06:10 PM UTC+4, essab...@gmail.com wrote:
> I get accuracy up to 99.9593% on the validation set as indicated in the command line. However, when I try the network on the validation set using "detect", the results are about 10%.
>
> What could be the reason for that?

I gave up and used another tool

[木兰香]

What's the name of another tool? Does it work well?

------------------ 原始邮件 ------------------

发件人: "essabasaeed";<essab...@gmail.com>;

发送时间: 2014年9月4日(星期四) 下午4:43

收件人: "eblearn"<ebl...@googlegroups.com>;

抄送: "essabasaeed"<essab...@gmail.com>;

主题: Re: Actual "detect" accuracy not matching validation accuracy

--
You received this message because you are subscribed to a topic in the Google Groups "eblearn" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/eblearn/yabF-xRz-LU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to eblearn+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.