MI vs TE in IDTxl

[Kate Dembny]

Hi all! 

I'm working on a project that utilize both mutual information (MI) and transfer entropy (TE) calculations. My understanding of the difference between TE and MI is related to the use of conditional mutual information for TE that allows for directed connections to be detected. However, it would seem in this toolbox uses CMI for both TE and MI calculations. Could someone clarify the difference between MI and TE in this toolbox for me if they both seem to be using CMI?

All the best,

Kate

[p.wol...@gmail.com]

Hi Kate,

Your understanding is correct, both with regards to the difference between TE and MI, and that IDTxl uses the CMI estimator in both cases. 

The difference lies in the variables that enter the (conditional) MI estimation. Loosely speaking, when estimating the multivariate TE between a source X and target Y, the toolbox calculates I(X-,Y+ | {Z,Y-}), i.e., the CMI between X’s past and Y’s future, conditional on Y’s past and the past of other relevant sources in the network, Z. The algorithm optimises Z by iteratively including past variables from nodes in the network other than Y and X. 

When estimating the (conditional) MI, the toolbox estimates I(X,Y+ | Z). Again, the algorithm optimises the conditioning set Z. In contrast to the TE, the target’s past is not included in the conditioning set and it should be possible to quantify instantaneous effects between X and Y (i.e., without any time lag, while the TE always has a lag of at least 1 time step). 

This is roughly how the two algorithms are implemented. Let me know if you have further questions.

Best,

Patricia