How to discussion the nature (e.g., positive or negative) of associations of transfer entropy

[Yorgo Hoebeke]

Dear all,

If transfer entropy from variable X to variable Y is high, it means that knowing the past states of X helps to predict the future states of Y. However, it doesn't tell us whether an increase in X predicts an increase in Y (positive association), or an increase in X predicts a decrease in Y (negative association). It simply indicates the predictive power from X to Y, without detailing the form of the relationship.

Is there a way, based on information theory, to explore the nature of this relationship? Or must we use other statistical methods such as correlation and regression to approximate the positive or negative nature of associations, with the limitation that it might not take into account non-linear effects?

[Joseph Lizier]

Information theory computes the amount of information in the association, but does not tell you the semantics of what that association is.

You could:

1. Interrogate the underlying probability distributions to see this in full detail, or

2. add statistics  like correlation as you say,

--joe
+61 408 186 901 (Au mobile)

--
You received this message because you are subscribed to the Google Groups "IDTxl" group.
To unsubscribe from this group and stop receiving emails from it, send an email to idtxl+un...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/idtxl/7e5b84d0-7f03-4929-bdbb-c842444a3c29n%40googlegroups.com.

[Manolo Martínez]

Couldn't one calculate pointwise mutual information/transfer entropy between the different bins?

Manolo 

To view this discussion on the web, visit https://groups.google.com/d/msgid/idtxl/CAL81BvSJPukexqgw92HXQXvA_b0kXe_U3j%3D2%3D2%3D_PwQkRbaYZg%40mail.gmail.com.

[Joseph Lizier]

Yes, thanks Manolo - looking at the pointwise/local values will definitely help here. I hesitated writing this before as it doesn't directly tell you the form of the relationship as per the enquiry, but it goes a long way to it (and you know I love these measures).

I've got some longer code examples doing this in preparation for distribution with my video lectures for the underlying JIDT. In the meantime, there are lots of published out there of using the local/pointwise values in this manner, e.g. our work on cellular automata, cat visual cortex, or in a slightly different fashion this example on heart-breath interaction.

--joe
+61 408 186 901 (Au mobile)

[Yorgo Hoebeke]

Thanks a lot for your answers! And thank you for the references.

In this paper, I went for simple correlations, since it was getting quite long already.

However,  I'd be keen to use pointwise/local values if I can get my hands on a bigger dataset of psychological time-series.
Mine only has 2184 replications; having +10 000 would be better. 

- Yorgo