Leave-one Out validation in matlab.

Maicon Kist

unread,

May 20, 2012, 5:42:36 PM5/20/12

to

Hi list,

I'm trying to implement a NN in Matlab using the 'leave one out' validation method.

My code is as follows:

====
net = patternnet(10);
net = configure(net, input, target);

net.trainParam.epochs = 1000;
net.divideMode = 'none';
net.divideFcn = '';

net.showWindow = false;
net.showCommandLine = true;

net = train(net, input, target);
resp = net(single_elem); % A element of the data set that wasnt used in train.
====

The problem is that the results are terrible compared to when "train" is used with the 'divideFcn" function.

Any suggestions?

Thanks.

Stephen Wolstenholme

unread,

May 21, 2012, 5:08:25 AM5/21/12

to

I don't use matlab so I can only consider what could cause your
problem in my software. The problem would almost certainly be due to
fact that one or more leave out example is not in the training range.

Steve

--
Neural Network Software. http://www.npsl1.com
EasyNN-plus. Neural Networks plus. http://www.easynn.com
SwingNN. Forecast with Neural Networks. http://www.swingnn.com
JustNN. Just Neural Networks. http://www.justnn.com

Maicon Kist

unread,

May 21, 2012, 7:28:47 AM5/21/12

to st...@npsl1.com

Hi Stephen.

I'm training the NN with a dataset given by my professor. When I train the NN using the default configuration for a NN in Matlab(it uses 70% of all data for training, 15% for validation and other 15% for test) the results are very good, around 85% of correct answers(it's a very hard problem).

Maybe I have to do something different in my code.

Thanks for you reply.

Greg Heath

unread,

May 27, 2012, 4:46:13 PM5/27/12

to

What are you trying to classify?
How many classes? c=?
How many input/output examples per class? N(i) = ?, i=1:c
Dimensionality of input vectors? I = ?
Are output vectors columns of the unit matrix eye(c)?

What class error rates do you get with the wine data set?

close all, clear all, clc,

[ x t ] = wine_dataset;
[ I N ] = size( x ) % 13 178
[ O N ] = size( t ) % 3 178
whos

Hope this helps.

Greg

Greg Heath

unread,

May 28, 2012, 9:16:40 AM5/28/12

to

On May 20, 5:42 pm, Maicon Kist <maiconk...@gmail.com> wrote:

1. Add comp.soft-sys.matlab to the newsgroup list.
2. Replace the commands

net.divideMode = 'none';
net.divideFcn = '';

with

net.divideFcn = 'dividetrain';
net.divideMode = 'sample'; % Default

Hope this helps.

Greg

Maicon Kist

unread,

May 30, 2012, 7:54:40 AM5/30/12

to

Hi greg

Hi Greg,

thanks for the hints. I will try it later, after my work.

And answering your questions:

> What are you trying to classify?

I'm training the the network with all my dataset, except one of them that I use for validation.
What I am doing is basically: remote one element of the dataset; train the NN with all the others; validate the NN with the removed element; repeat for other elements.

> How many classes? c=?
Eight classes.

How many input/output examples per class? N(i) = ?, i=1:c

336 input
143/336 are of class 1
77 /336 are of class 2
2 /336 are of class 3
2 /336 are of class 4
35 /336 are of class 5
20 /336 are of class 6
5 /336 are of class 7
52 /336 are of class 8

> Dimensionality of input vectors? I = ?

7 characterists, all of them in the range [0..1], determine the class of each element of the dataset

> Are output vectors columns of the unit matrix eye(c)?

Columns: [8x1]

> What class error rates do you get with the wine data set?

Didn't test yet. Later I post the results here.