Re: ANN: SwingNN forecasting
Phil Sherrod
On 17-Feb-2008, Steve Wolstenholme <st...@tropheus.demon.co.uk> wrote:
> Your data is imported into a grid and used to train a neural network.
> The input values are forced to swing beyond their limits. The output
> values are forecasted by the neural network. A new neural network is
> created using the new inputs and forecasted outputs. The neural
> networks are compared. The inputs are adjusted and another new neural
> network is created. The process continues until a new neural network
> agrees with the original about the forecasts. The forecasts are added
> to the grid for you to use.
I don't understand. If the input values are forced to swing beyond their
limits, how do you know what the output value would be that corresponds to the
artificial inputs?
Let's say we have a training data set with x,y value pairs generated from
y=2+3*x over the domain x=(0,10). How does your method produce a network that
can predict y when x=100?
--
Phil Sherrod
(PhilSherrod 'at' comcast.net)
http://www.dtreg.com (Decision trees, Neural networks, SVM and Genetic
modeling)
http://www.nlreg.com (Nonlinear Regression)
Phil Sherrod
On 17-Feb-2008, Steve Wolstenholme <st...@tropheus.demon.co.uk> wrote:
> We all know that neural networks are useless at extrapolating so what
> I do is a lot of validating. New networks are produced and trained
> using out of range values.
What is the target (dependent) variable value used for the training with
out-of-range input values?
> They are validated using in range values.
> If a network gives very similar results to the original the process
> stops. Every new network increases the risk level. It is a "guess &
> test" process. Obviously accuracy can't be guaranteed. If it could I
> would have made my first billion about 20 years ago.
I would like to see some examples of how effective your method is. How about
running these examples and posting the results:
1. Train the network on y=2+3*x using perfect data with 100+ points over
x=(0,10), then show us the results with x extrapolated out to 20, 50, 100, 500,
etc. I want to see how far it goes before it starts to diverge wildly.
2. Train the network on y=sin(x) over x=(0,2pi) then extrapolate out and see if
it follows a sine wave.
Phil Sherrod
On 17-Feb-2008, Steve Wolstenholme <st...@tropheus.demon.co.uk> wrote:
> >What is the target (dependent) variable value used for the training with
> >out-of-range input values?
>
> Out-of-range values are generated by extrapolating any trends found in
> the in-range examples.
And how do you do that extrapolation? It sounds like you have a recursive
problem: you are using extrapolation to train a NN to do extrapolation. Are
you using linear, nonlinear or auto-regression to extrapolate the trend? I
don't see how the trained NN could be more accurate than the training
extrapolation method. If the training extrapolation method is good, who needs
a NN?
> >1. Train the network on y=2+3*x using perfect data with 100+ points over
> >x=(0,10), then show us the results with x extrapolated out to 20, 50, 100,
> >500,
> >etc. I want to see how far it goes before it starts to diverge wildly.
> >
> >2. Train the network on y=sin(x) over x=(0,2pi) then extrapolate out and see
> >if
> >it follows a sine wave.
>
> OK. Will do, but it may take a few days. I'm a but busy just now.
OK, that's fine. Also please add this to the test:
3. Train y = 10 + 2*x + 0.1x^2 over x=(0,10) and extrapolate it. I want a
non-oscillating but nonlinear example.
Greg Heath
wrote:
> SwingNN makes forecasting and prediction easy.
What is the difference between forecasting and prediction?
Is this method exclusive to time-series?
How are inputs normalized?
Can the output be multidimensional?
> Import data from text, csv, spreadsheet, image or binary files.
>
> Your data is imported into a grid and used to train a neural network.
Which is associated with some baseline error.
Are the number of hidden nodes minimal?
> The input values are forced to swing beyond their limits.
Using the original normalization?
One at a time or in random combinations?
If the latter, are the input correlations of the original maintained?
How much beyond? Within a certain percentage or
within a certain fraction of the stdev?
> The output
> values are forecasted by the neural network. A new neural network is
> created using the new inputs and forecasted outputs.
Using the same number of hidden nodes?
> The neural
> networks are compared. The inputs are adjusted and another new neural
> network is created.
How many adjustments per input for each new design?
In the spirit of "jittering", it seems that surrounding each
multidimensional input with a small elliptically shaped
cloud that is oriented to maintain input correlations might
have merit. Then you only have to change the size of
the cloud, not the shape.
New network designs can be combined with previous
designs to form a committee.
> The process continues until a new neural network
> agrees with the original about the forecasts.
If your output is not piecewise continuous, doesn't
"agreement" mean to within some specified threshold?
> The forecasts are added to the grid for you to use.
I would trust the forcasts of an evolved committee more
than the forecasts of a single evolved network.
Hope this helps.
Greg
Phil Sherrod
On 19-Feb-2008, Steve Wolstenholme <st...@tropheus.demon.co.uk> wrote:
> >> >1. Train the network on y=2+3*x using perfect data with 100+ points over
> >> >x=(0,10), then show us the results with x extrapolated out to 20, 50,
> >> >100,
> >> >500,
> >> >etc. I want to see how far it goes before it starts to diverge wildly.
>
> It can't go that far because it terminates when the risk level is too
> high.
>
> y is correct up to x = 22. I have no idea what happens then.
>
> >> >
> >> >2. Train the network on y=sin(x) over x=(0,2pi) then extrapolate out and
> >> >see
> >> >if
> >> >it follows a sine wave.
>
> It follows a sine wave up to 3pi. It won't work with more oscillations
> until the next release.
>
> >
> >OK, that's fine. Also please add this to the test:
> >
> >3. Train y = 10 + 2*x + 0.1x^2 over x=(0,10) and extrapolate it. I want a
> >non-oscillating but nonlinear example.
>
> y is correct when the run terminates on risk at x = 19
Thanks for posting the results Steve. The extrapolations went further than I
expected.
Greg Heath
wrote:
> On Sun, 17 Feb 2008 19:29:21 GMT, "Phil Sherrod"
>
> <PhilSher...@REMOVETHIScomcast.net> wrote:
-----SNIP
> >> >2. Train the network on y=sin(x) over x=(0,2pi) then extrapolate out and > >> > see if it follows a sine wave.
>
> It follows a sine wave up to 3pi. It won't work with more oscillations
> until the next release.
I'm surprised that the trend reversal at the local max 5*pi/2
is successfully negotiated. I'm also puzzled since the correct
outputs are unavailable for training.. Can you explain
what is going on there?
Greg
Greg Heath
wrote:
> On Tue, 19 Feb 2008 23:19:06 -0800 (PST), Greg Heath
> next version should be able to use regular reversals.
Sorry, I still don't understand. The original model is trained on
data
from 0 to 2*pi. The next models are trained on 0-2*pi data and
2*pi to 2*pi + x extrapolations. I can see, I think, how the
extrapolation
can succeed for x <= pi/2. However, at 5*pi/2, the true mapping
goes thru a local max and begins to decrease.
How does the latest extrapolation model know this?
Hope this helps.
Greg
Greg Heath
wrote:
> On Thu, 21 Feb 2008 07:40:32 -0800 (PST), Greg Heath
But you said it follows the sine wave to 3*pi.
Slip of the tongue?
Greg