number of input (variables)
amir
input variable.what would it effect the number of neuron in hidden
layer or the number of hidden layer itself? Say we could produce
enough data for training.
Greg Heath
> Is there a limitation in number of input (variables).
Given a set of output variables there is usually a limit to the
number of necessary inputs.
Whether or not those inputs or an alternative set are available
is one issue.
Other issues are:
Whether or not a given set of inputs contains enough information
to yield the desired outputs.
If the given set of inputs is sufficient, but not minimal, should
time be spent finding and removing redundant and/or irrelevant
inputs or should an attempt be made to design a net which will
automatically ignore them.
> Say we have 100
> input variable.what would it effect the number of neuron in hidden
> layer or the number of hidden layer itself? Say we could produce
> enough data for training.
Usually one hidden layer is sufficient.
If training to convergence without regularization, the number of
hidden
nodes is usually determined by trial and error with the constraint
that
the number of training equations, Neq, is sufficiently large for
estimating the Nw unknown weights without overfitting.
If using an overfitting mitigation technique (e.g., weight decay
and/or Early Stopping) Neq can be significantly smaller than
when training to convergence.
These issues are thoroughly discussed in the FAQ.
Also, see my post on pretraining advice.
A Google Group search using keywords
greg-heath Neq Nw
may be helpful.
Hope this helps.
Greg
Oxygen
> On Oct 23, 11:03 pm, amir <beh.am...@gmail.com> wrote:
>> Is there a limitation in number of input (variables).
>
> Given a set of output variables there is usually a limit to the
> number of necessary inputs.
> Whether or not those inputs or an alternative set are available
> is one issue.
> Other issues are:
> Whether or not a given set of inputs contains enough information
> to yield the desired outputs.
> If the given set of inputs is sufficient, but not minimal, should
> time be spent finding and removing redundant and/or irrelevant
> inputs or should an attempt be made to design a net which will
> automatically ignore them.
Amir should do some reading on variable selection. It depends on
what you are trying to model and how much data you have.
Greg's comments cover this conceptually.
You need to explore the alternatives and measure their contribution
to prediction error.
For simple models (Linear Models or GLMs) there used to be a rule
that if you used more than the five most important predictors, you needed
a good argument.
With NNs or hierarchical models or SVMs, this doesn't apply any more.
Just be careful to manage co-linearity.
A hybrid approach is to project the first five principle components (PCs)
and use them as inputs to build a model. Then test each variable for its
ability to predict the residual for a model built with the first five PCs. Then
throw a few of the most important of those in the mix of inputs...
O.
Greg Heath
>
> A hybrid approach is to project the first five principle
> components (PCs) and use them as inputs to build a model.
Good for regression but not necessaily for classification.
Think of two parallel cigar-shaped distributions in 2-D.
The dominant PC along the cigars is orthogonal to the
direction of separation.
> Then test each variable for its
> ability to predict the residual for a model built with the first five
> PCs. Then throw a few of the most important of those in the
> mix of inputs...
In classification, the most effective variables tend to
dominate in linear and quadratic classifier models.
Consider these first instead of PCs.
Hope this helps.
Greg