Ragwitz' criterion and Ragwitz & Kantz 2002

[Degang WU]

In the other thread we talked about the guideline for choosing suitable embedding dimension and embedding delay, and Ragwitz' criterion was brought up.

One technical detail that is bothering me is that when we optimize the relative prediction error we have to choose the number of nearest neighbors for the locally constant predictor. Is the relative prediction error sensitive to the choice of that number?

I have been trying to reproduce some of the results in Ragwitz and Kantz 2002 mentioned by Michael Wibral in the other thread:

Markov models from data by simple nonlinear time series predictors in delay embedding spaces.
Ragwitz M, Kantz H.
Phys Rev E Stat Nonlin Soft Matter Phys. 2002 May;65(5 Pt 2):056201. Epub 2002 Apr 15.

I have been working hard reproducing Fig. 3 in Ragwitz & Kantz 2002 (Fig. 3 is one of the key results in the work) but so far I am not able to reproduce it:

1)  I used TRENTOOL's TEragwitz.m to calculate the relative prediction error as a function of embedding dimension and embedding delay, and I have tried a range of sizeNei (the number of nearest neighbors used in the locally constant predictor) and the length of the time series, but the results were very different from Ragwitz & Kantz 2002.

2) Ragwitz & Kantz 2002 is very vague in how they calculated the relative prediction error. Instead of fixing the number of nearest neighbors, they fixed the radius of neighborhood and went through some pretty elaborate procedures if the number of points in the neighborhood is too small.

3) As a last resort, I tried Tisean's lzo-test (http://www.mpipks-dresden.mpg.de/~tisean/Tisean_3.0.0/docs/docs_c/lzo-test.html). Setting k=30 (the minimal number of nearest neighbors used for the locally constant predictor) seemed to produce something similar to Ragwitz & Kantz 2002's Fig 3(a), but I couldn't reproduce anything similar to FIg. 3(b) and 3(c). What's worse, the relative prediction error seemed to be quite sensitive to the value of k.

I'm just wondering if anyone has successfully reproduce the results in Ragwitz & Kantz 2002.

[Joseph Lizier]

HI Degang,

Yes, the relative prediction error is certainly going to change depending on the number of nearest neighbours that you use. With that said, I'm not familiar with studies of precisely how that looks.

In JIDT, the Ragwitz criteria is only implemented (so far) for the KSG estimator, and so I conveniently default it to use the same number of nearest neighbours that KSG uses (usually K=4) though that can be overriden.

I haven't tried to reproduce the Ragwitz and Kantz results though, so I can't really comment on that.
Regarding what comes out of TRENTOOL's TEragwitz.m, I wonder if TRENTOOL is already operating on the normalised data (as is the Ragwitz predictor in JIDT) whereas the results in Ragwitz may be on raw data? This has the capacity to change the nearest neighbours being selected. Perhaps Michael / Patricia can comment.

--joe
+61 408 186 901 (Au mobile)

--
JIDT homepage: http://code.google.com/p/information-dynamics-toolkit/
---
You received this message because you are subscribed to the Google Groups "Java Information Dynamics Toolkit (JIDT) discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jidt-discuss...@googlegroups.com.
To post to this group, send email to jidt-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/jidt-discuss.
To view this discussion on the web visit https://groups.google.com/d/msgid/jidt-discuss/5a5a50a1-ffb8-48b3-9171-4c4521ce93a5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Degang WU]

Hi Joe,

I'm still not able to reproduce the Ragwtiz Kantz 2002. The method employed in Ragwitz & Kantz 2002 is rather elaborate and they used Euclidean distance rather than maxnorm. Waiting for comments from Michael / Patricia.

[Joseph Lizier]

Hi Degang,

The use of different norms could certainly explain it.

If you're really keen for a comment from Michael/Patricia on the use of TEragwitz.m from TRENTOOL for that, you might be best served following up in the TRENTOOL support channels.

--joe
+61 408 186 901 (Au mobile)

To view this discussion on the web visit https://groups.google.com/d/msgid/jidt-discuss/82a4f3e8-aba1-4718-b1b4-7ee1e0227ebf%40googlegroups.com.

[Michael Wibral]

Hi Degang,

I agree with what Joe said. As far as TEragwitz.m in TRENTOOL is concerned, it's minimizing the storage at conditions that are typical for TE analysis (i.e. just 4 neighbours, and using the maxnorm).

Best,
Michael

To view this discussion on the web visit https://groups.google.com/d/msgid/jidt-discuss/CAL81BvRQ7G6rX%2BjO6yh6uQf%3DObvi_Ug1cmpbhEZVJ1FgSbodhw%40mail.gmail.com.

[Degang WU]

Hi Michael,

Since you pointed me to Ragwitz & Kantz 2002 in the other thread, I wonder if you and/or your students have ever reproduced their results (Fig.3 especially). That paper is pretty interesting in itself, so it would be a shame if the results cannot be reproduced.

Degang

You received this message because you are subscribed to a topic in the Google Groups "Java Information Dynamics Toolkit (JIDT) discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jidt-discuss/P88cxpN20ts/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jidt-discuss...@googlegroups.com.

To post to this group, send email to jidt-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/jidt-discuss.

To view this discussion on the web visit https://groups.google.com/d/msgid/jidt-discuss/55F7E455.50203%40em.uni-frankfurt.de.