any problem with DIC

[Juan]

Hi, list experts,

According to the demo, the DIC and Deviance are positive values. For my data, I got negative values, such as DIC: -15295.957170; deviance: -15349.036299.

Would this be a signal that I made mistakes anywhere?

Thanks in advance

[Thomas Wiecki]

Hi,

No, DIC can be positive or negative. The absolute value is hard to interpret so only look at the difference.

Thomas

--
You received this message because you are subscribed to the Google Groups "hddm-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hddm-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Thomas Wiecki, PhD

Data Science Lead, Quantopian Inc, Boston

[Juan]

Then does it still mean that the lower DIC means a better fit?

For instance:  Model with DIC: -1600 is better than the model with DIC: -1500?

Thanks

[Thomas Wiecki]

Yes, that's right.

[Chuan-Peng Hu]

Hi, Thomas,

Thank you for your efforts in maitaining this forum!

I am new to HDDM, and one question is puzzling to me: is it case that the more parameters are free to vary, the lower DIC is?

I read two recent papers (please see below, one of them is yours ;-) ) using HDDM, both compared different models in which different parameters were allowed to vary, both showed that the more parameters were free to vary, the lower DIC was. Also, I used HDDM to fit my own experimental data ( 27 participants, 350 trials per participant), and also found that the more paramters were free to vary, the lower DIC was.

Model	Free to vary	DIC
1	v, a, Ter, z	-800
2	v, Ter, z	-480
3	v, a, Ter	-266
4	a, Ter, z	144
5	v, a, z	630
6	v, Ter	1166
7	Ter, z	1216
8	v, z	1688
9	a, Ter	2179
10	v,a	2750
11	a, z	2962
12	Ter	2415
13	v	2955
14	z	2977
15	a	4292

If this is the case, then how to chose the parameters to be free to vary? 

Thanks in advance!

Reference:

Frank, M. J., Gagne, C., Nyhus, E., Masters, S., Wiecki, T. V., Cavanagh, J. F., & Badre, D. (2015). fMRI and EEG Predictors of Dynamic Decision Parameters during Human Reinforcement Learning. The Journal of Neuroscience, 35(2), 485-494. doi:10.1523/jneurosci.2036-14.2015

Nie, Q.-Y., Maurer, M., Müller, H. J., & Conci, M. (2016). Inhibition drives configural superiority of illusory Gestalt: Combined behavioral and drift–diffusion model evidence. Cognition, 150, 150-162. doi:10.1016/j.cognition.2016.02.007

[Samuel Mathias]

To understand DIC, please consult the Wikipedia article: https://en.m.wikipedia.org/wiki/Deviance_information_criterion

Generally, if the additional parameters explain more features in the data, DIC will go down. If the additional parameters are not useful, DIC will go up. DIC is therefore a measure of goodness of fit penalized for model complexity. Note however that many authors consider DIC to be too generous to over-complex models. It's a good idea to follow up DIC with other tests, e.g. a PPC.

--
Samuel R. Mathias, Ph.D.
Associate Research Scientist (ARS)
Neurocognition, Neurocomputation and Neurogenetics (n3) Division
Yale University School of Medicine
40 Temple Street, Room 694
New Haven CT 06510
http://www.srmathias.com

[Chuan-Peng Hu]

Hi, Samuel, 

Thank you very much for you reply and the link!