Ragwitz auto-embedding in Conditional Transfer Entropy

[Aleksander Janczewski]

Dear Professor Lizier,

I am unsure how to perform Ragwitz auto embedding for conditional transfer entropy.

I am performing an information-theoretic analysis between 7 variables. The plan is as follows: 

1. First, I estimate Transfer entropy using Kraskov estimator Algorithm1 between each possible pair of variables (the network). The history length and embedding delays are determined automatically using the Ragwitz criterion. After history length and embedding delays are set, I search for a delay from source to destination that maximizes the Transfer Entropy. After performing the above for each source-destination pair, I have two sets of optimal parameters for each variable, each specific to the source-destination pair: one when the variable is considered as a source and one when the variable is considered as a destination, different for every source-destination combination. And finally, I also perform a permutation test to determine the significance of my estimates. 

2. Next, I plan to estimate the conditional transfer entropy, again using Kraskov estimator Algorithm1 between each pair of the variables. The goal is to determine whether the transfer of information between source and destination is significant, even if it is conditioned on all the other statistically significant sources to that destination.

The question that I have here is as follows:

- Since I have already determined the optimal history length and embedding delays and delay from source to destination for each source-destination pair of variables while estimating the transfer entropy, can I use these parameters in my estimation for conditional transfer entropy? To do so, I would treat each conditional variable as another source, hence I would use a set of optimal parameters that I obtained when auto embedding this variable as a source. 

What about the parameters for the destination variable? Do I need to redo Ragwitz auto embedding of the destination and somehow account for all conditional variables as other sources? If that's the case, could you help me clarify the recipe for the auto embedding of destination for conditional transfer entropy?

Thank you very much for your time and help.

With Best Regards,

Aleksander Janczewski

[Joseph Lizier]

Dear Aleksander,

In short yes you can and should use (some of) the parameters you've previously determined from the pairwise calculations for the conditionals:

For the target, you should read out the embedding parameters that were determined from the pairwise calculations (after the calculation is made) by calling, for example in Matlab:

dest_k = teCalc.getProperty('k_HISTORY');
dest_tau = teCalc.getProperty('k_TAU');

Then supply them directly to the conditional TE calculator:

teCalc.setProperty('k_HISTORY', dest_k);
teCalc.setProperty('k_TAU', dest_tau);

For the conditional (assuming just one for now), you're right that you would use the parameters determined from when it was a source. So similarly, you would read them out and store them first from the relevant pairwise calculation:

source_as_conditional_k = teCalc.getProperty('l_HISTORY');
source_as_conditional_tau = teCalc.getProperty('l_TAU');

source_as_conditional_delay = teCalc.getProperty('DELAY');

Then supply them directly to the conditional TE calculator:

condTeCalc.setProperty('COND_EMBED_LENGTHS', source_as_conditional_k);
condTeCalc.setProperty('COND_TAUS', source_as_conditional_tau);

condTeCalc.setProperty('COND_DELAYS', source_as_conditional_delay);

Note that these properties for conditionals are expecting one value for every conditional variable, so if you're using multiple conditionals you would supply them as a comma separated string of integers. (In the same order that the conditionals are supplied in).

For the source, you can use the same Ragwitz embedding that you determined from the pairwise calculation and then pass those in

source_k = teCalc.getProperty('l_HISTORY');
source_tau = teCalc.getProperty('l_TAU');

Then supply them directly to the conditional TE calculator:

condTeCalc.setProperty('l_HISTORY', source_k);
condTeCalc.setProperty('l_TAU', source_tau);

However, I would suggest that you manually check for the delay that maximises the conditional TE from this source now, since this may be different with the conditioning.

This is not automated in JIDT.

Along these lines, I would suggest that instead of directly conditioning on all sources, you would be best to iteratively determine the strongest source first, then condition on that whilst selecting the next strongest source, and so on until the sources are no longer statistically significant. This is because directly going to a calculation of conditional TE with 5 conditionals, all embedded, is likely to exhaust the statistical power in many realistically sized data sets.

What I am getting at here is the higher level greedy / iterative algorithm that is implemented in our partner IDTxl toolkit, and described in e.g. this paper.

Whilst I am planning to add a simplistic implementation of what I've described here to be distributed with JIDT, that will be for purposes of demonstration -- the IDTxl implementation is that which does this properly, e.g. with non-uniform embeddings rather than Ragwitz.

Hope that helps,

--joe

--
JIDT homepage: https://github.com/jlizier/jidt
---
You received this message because you are subscribed to the Google Groups "Java Information Dynamics Toolkit (JIDT) discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jidt-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jidt-discuss/d06eff11-6cbb-469b-8341-431e0ee6e6a9n%40googlegroups.com.