Training NN with outputs not with vectors but matrices
Takis Pan
I would appreciate if anyone could give me an answer to this problem. I was wondering if a MLP neural network could be trained giving at the output layer not a vector (e.g. 4x1, which means 4 output nodes) but a matrix (e.g. 4x4). If this is possible, how many output nodes should I declare at the training of the network. For example I want the output of the network to be four pairs of x,y, namely, x1,y1 - x2,y2 - x3,y3 - x4,y4. I should note that x,y are independent variables.
Thank you in advance,
Takis
Greg Heath
> Hello,
>
> I would appreciate if anyone could give me an answer to this problem. I was wondering if a MLP neural network could be trained giving at the output layer not a vector (e.g. 4x1, which means 4 output nodes) but a matrix (e.g. 4x4). If this is possible, how many output nodes should I declare at the training of the network. For example I want the output of the network to be four pairs of x,y, namely, x1,y1 - x2,y2 - x3,y3 - x4,y4. I should note that x,y are independent variables.
Your question is not clearly stated. So I will just state:
There is one output node for each output signal. If your
output, y, is a time series of an mXn matrix for N observation
times and you want to predict the signal for time t(N+1).
Unfold the training targets if size(t) = [m n N], use the
matrix unfolding operator (:) to reduce the matrices to
vectors then size(t1) = [p N] for p = m*n and the rest
folloes as usual.
Therefore the net output will have p = m*n nodes and
the matrix structure of the output y can be recovered
from y1 by using the reshape function.
Hope this helps.
Greg
Takis Pan
Generally, a MLP has an output layer with N nodes, with each node correspond to a single value. Thus, one can represent these values with a vector, let say T. If the corresponding input vector is P, then the training pairs are [P,T] and the training could be:
net=newff(minmax(P),[30,N],{'tansig','purelin'},'trainlm');
net=init(net);
[pn,minp,maxp,tn,mint,maxt] = premnmx(P,T);
[net,tr]=train(net,pn,tn);
where 30 is the number of hidden layer's nodes and N the number of output nodes.
Now, I do not have the case of a time series. I want to examine the case where each output node gives a point in the xy plane, corresponding to a coordinate (e.g. x1, y1). How could I train a NN, given the corresponding input vector P, to find N such points (xn, yn), namely N pairs of x,y? What number of output nodes should I give in the line:
net=newff(minmax(P),[30,?],{'tansig','purelin'},'trainlm'); ? The output would be a matrix (Nx2), and if so is it possible to use as an output a matrix instead of a vector?
Probably the information you gave about unfolding the targets is useful, but I am not familiar with this procedure, and I am not sure I comprehend it properly.
Thank you for your patience and I hope I made myself more clear this time.
Takis
Greg Heath <he...@alumni.brown.edu> wrote in message <30f3ffdd-9a01-4897...@p23g2000vbl.googlegroups.com>...
Greg Heath
INTERSPERSED WITHIN THE PREVIOUS POST
On Nov 9, 2:33 pm, "Takis Pan" <k_g...@hotmail.com> wrote:
> Dear Greg, thank you for your quick response. Sorry if my question was misleading. Let me try be more precise about what I need.
>
> Generally, a MLP has an output layer with N nodes, with each node correspond to a single value. Thus, one can represent these values with a vector, let say T. If the corresponding input vector is P, then the training pairs are [P,T] and the training could be:
> net=newff(minmax(P),[30,N],{'tansig','purelin'},'trainlm');
> net=init(net);
> [pn,minp,maxp,tn,mint,maxt] = premnmx(P,T);
> [net,tr]=train(net,pn,tn);
> where 30 is the number of hidden layer's nodes and N the number of output nodes.
>
> Now, I do not have the case of a time series. I want to examine the case where each output node gives a point in the xy plane, corresponding to a coordinate (e.g. x1, y1). How could I train a NN, given the corresponding input vector P, to find N such points (xn, yn), namely N pairs of x,y? What number of output nodes should I give in the line:
> net=newff(minmax(P),[30,?],{'tansig','purelin'},'trainlm'); ?
size(t,1) = 2*N
>The output would be a matrix (Nx2),
No. The output for a single input must be a vector of size [2*N 1]
and if so is it possible to use as an output a matrix instead of a
vector?
No. Unfold the matrix into a vector using (:).
> Probably the information you gave about unfolding the targets is useful, but I am not familiar with this procedure, and I am not sure I comprehend it properly.
A = rand(2,3)
B = A(:)
Hope this helps.
Greg
> Greg Heath <he...@alumni.brown.edu> wrote in message <30f3ffdd-9a01-4897-b605-37d1b9544...@p23g2000vbl.googlegroups.com>...
> > On Nov 9, 8:23?am, "Takis Pan" <k_g...@hotmail.com> wrote:
> > > Hello,
>
> > > I would appreciate if anyone could give me an answer to this problem. I was wondering if a MLP neural network could be trained giving at the output layer not a vector (e.g. 4x1, which means 4 output nodes) but a matrix (e.g. 4x4). If this is possible, how many output nodes should I declare at the training of the network. For example I want the output of the network to be four pairs of x,y, namely, x1,y1 - x2,y2 - x3,y3 - x4,y4. I should note that x,y are independent variables.
>
> > Your question is not clearly stated. So I will just state:
>
> > There is one output node for each output signal. If your
> > output, y, is a time series of an mXn matrix for N observation
> > times and you want to predict the signal for time t(N+1).
>
> > Unfold the training targets if size(t) = [m n N], use the
> > matrix unfolding operator (:) to reduce the matrices to
> > vectors then size(t1) = [p N] for p = m*n and the rest
> > folloes as usual.
>
> > Therefore the net output will have p = m*n nodes and
> > the matrix structure of the output y can be recovered
> > from y1 by using the reshape function.
>
> > Hope this helps.
>
> > Greg- Hide quoted text -
>
> - Show quoted text -
Takis Pan
> PLEASE DO NOT TOP POST. PLACE YOUR REPLY AFTER OR
> INTERSPERSED WITHIN THE PREVIOUS POST
>
> > Dear Greg, thank you for your quick response. Sorry if my question was misleading. Let me try be more precise about what I need.
> >
> > Generally, a MLP has an output layer with N nodes, with each node correspond to a single value. Thus, one can represent these values with a vector, let say T. If the corresponding input vector is P, then the training pairs are [P,T] and the training could be:
> > net=newff(minmax(P),[30,N],{'tansig','purelin'},'trainlm');
> > net=init(net);
> > [pn,minp,maxp,tn,mint,maxt] = premnmx(P,T);
> > [net,tr]=train(net,pn,tn);
> > where 30 is the number of hidden layer's nodes and N the number of output nodes.
> >
> > Now, I do not have the case of a time series. I want to examine the case where each output node gives a point in the xy plane, corresponding to a coordinate (e.g. x1, y1). How could I train a NN, given the corresponding input vector P, to find N such points (xn, yn), namely N pairs of x,y? What number of output nodes should I give in the line:
> > net=newff(minmax(P),[30,?],{'tansig','purelin'},'trainlm'); ?
>
> size(t,1) = 2*N
>
> >The output would be a matrix (Nx2),
>
> No. The output for a single input must be a vector of size [2*N 1]
>
> and if so is it possible to use as an output a matrix instead of a
> vector?
>
> No. Unfold the matrix into a vector using (:).
>
> > Probably the information you gave about unfolding the targets is useful, but I am not familiar with this procedure, and I am not sure I comprehend it properly.
>
> A = rand(2,3)
> B = A(:)
>
> Hope this helps.
>
> Greg
>
-------------------
Thank you again Greg,
I understand what you are proposing, unfortunately I had already tried with a vector of size[2*N,1], with no good results. I expected that probably there would not be any other alternative for the network output (a matrix instead of a vector), but I asked just in the case I was wrong.
I believe the results are not good because in the case of a vector you have a row of successive xs and ys, so the network does not learn to create pairs of x and y. It thinks that the two successive nodes correspond to independent values of the same variable. Probably it does not distinguishes between x and y. That's why I wanted to assign to each output node a pair of numbers (x and y) and not just one.
Thank you for your time and if you have another idea I will be happy to hear it.
Regards,
Takis
Greg Heath
> Greg Heath <he...@alumni.brown.edu> wrote in message <a8d9417e-3150-40c9-8ec2-9de855c65...@l35g2000vba.googlegroups.com>...
No, it does not.
It assumes whatever correlation structure is contained in the target
matrix.
> Probably it does not distinguishes between x and y.
It doesn't and it doesn't have to.
> That's why I
> wanted to assign to each output node a pair of numbers (x and y) and not just .
> one. Thank you for your time and if you have another idea I will be happy to
> hear it.
The cause of failure is not output vector formatting.
Does your training data adequately characterize the complete data set?
size(Ptrn) = ?
size(Ttrn) = ?
size(Ptst) = ?
size(Ttst) = ?
Are you overfitting with H = 30?
etc.
Greg
Takis Pan
>
> The cause of failure is not output vector formatting.
> Does your training data adequately characterize the complete data set?
> size(Ptrn) = ?
> size(Ttrn) = ?
> size(Ptst) = ?
> size(Ttst) = ?
> Are you overfitting with H = 30?
>
> etc.
>
> Greg
Dear Greg,
It is indeed possible that the number of training pairs is not adequately big. Till now I have only worked with typical feedforward networks. Do you think Self Organizing Networks could help me more with this particular problem?
Thank you,
Takis
Greg Heath
> Greg Heath <he...@alumni.brown.edu> wrote in message <c77481c4-1fd6-4c19-a9b7-20178f67f...@p36g2000vbn.googlegroups.com>...
I see you didn't answer my questions. What are the sizes?
Till now I have only worked with typical feedforward networks. Do you
think Self Organizing Networks could help me more with this particular
problem?
This particular problem looks pretty ordinary.
A self organizing nets is unsupervised.
Why would you think that would help?
Greg
Takis Pan
> >
> > > etc.
> >
> > > Greg
> >
> > Dear Greg,
> >
> > It is indeed possible that the number of training pairs is not adequately big.
>
> I see you didn't answer my questions. What are the sizes?
>
> Till now I have only worked with typical feedforward networks. Do you
> think Self Organizing Networks could help me more with this particular
> problem?
>
> This particular problem looks pretty ordinary.
> A self organizing nets is unsupervised.
> Why would you think that would help?
The problem is ordinary, however the great number of combinations and ambiguities make it difficult. I do not know much about self organizing nets, but maybe they could give a more clear picture and put in an order, at least the inputs of the problem.
Thank you,
Regards,
Takis
>
> Greg
>
Greg Heath
> Greg Heath <he...@alumni.brown.edu> wrote in message <70d526b2-26c4-4258-bd8a-e7aab174a...@l2g2000yqd.googlegroups.com>...
What does a single input vector represent?
What does the corresponding output vector represent?
Greg