Results Normalization

[Douglas Hull]

Hello Dr. Lizier and the team,  

I wonder if you could kindly suggest some normalization techniques that we can perform on the pairwise Transfer Entropy results so that the relative coupling strengths between the time series variables are retained. In our study, we have 79 nodes. Thank you very much for your assistance!

[Thomas Varley]

I'm not Joe, but I would suggest dividing the empirical TE by the conditional entropy rate of the receiving variable (using the same embedding as you do for the self-history in the TE calculation). A less-ideal (but much easier) option is to divide the TE value just by the Shannon entropy of the receiver (as was done in this paper: https://direct.mit.edu/netn/article/3/2/384/2220/Computation-is-concentrated-in-rich-clubs-of-local).

All the best
~Thomas

[Joseph Lizier]

Thanks Thomas, and:

> I'm not Joe

well played :).

Douglas -- Thomas' comments are spot on.

If you want to read further on how the TE is a component of the entropy of the next value of the target/receiver, and within that of the entropy rate of that target/receiver (this being a tighter bound as Thomas says), I can suggest sections 3.2.2 and 4.1.4 of my thesis/book (at Springer and preprint). Which you choose depends on which question you want to answer: if you want to know what proportion of the total uncertainty in the variable was accounted for by the TE, then normalise by entropy; if you want to know what proportion of the remaining uncertainty after accounting for the variable's own past was accounted for by the TE, then normalise by the entropy rate. The latter is more common because it's a tighter bound, but both address slightly different questions and carry their own meaning.

There's also some commentary on these as normalisation techniques at section 4.5.2 of our book on TE (Springer). There we also describe how making sure we're incorporating bias correction in the estimator is an important ingredient.

There are a couple of caveats to mention here:

1. The above only applies for Shannon entropies (i.e. for discrete or discretised variables). You cannot do this for continuous valued variables. This is because the entropy and entropy rate are differential entropies, which are not strictly non-negative and are not by definition upper limits to the TE (or any other mutual information involving the variable). So if you've estimated TE say via the KSG estimator, you're not going to be able to use the above. Indeed, I'm not sure there's a good answer for what you could do. You might try to compare the pairwise TE against the whole collective TE from all of the parent variables identified to the target, but this will necessarily leave out any intrinsic uncertainty from being included in the denominator, and I think that would introduce quite some variability.

2. You can't run the above normalisations directly in IDTxl as we don't ship an entropy estimator. You could post-process your IDTxl produced TEs with JIDT for this however

Hope that helps.

--joe

--
You received this message because you are subscribed to the Google Groups "IDTxl" group.
To unsubscribe from this group and stop receiving emails from it, send an email to idtxl+un...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/idtxl/7de34212-6d33-48a7-b141-71a989577f66n%40googlegroups.com.

[Thomas Varley]

While we're on this topic - does it make sense to normalize local information values in the same way? Could I divide the local transfer entropy of a given moment by the local entropy of the receiver? I feel like that should be allowable, although my intuitions on this could be off.

All the best
~Thomas