Negative DIC using Large Sample: > 200 trials and 500 people

[Frances Jin]

Hi HDDM experts,

I am fitting models using more than 500 subjects, with each having more than 200 trials (but imbalanced across conditions). My first try using 5000 samples, 500 burn in and 5 thin resulted in a large Negative DIC.

I understand that DIC can be negative, but I am not sure if I have used the best model set up, particularly given my large sample size.

Could someone let me know if the negative DIC is expected? Also, is my sampling, burn in, and thinning a reasonable choice? Any other things I need to pay attention to with large data?

Many thanks, in advance!

Best,

Frances

[Thomas Wiecki]

That's fine, as long as your chains converged. 

--
You received this message because you are subscribed to the Google Groups "hddm-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hddm-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sam Mathias]

I once used HDDM on a data set with ~2,000 subjects. It took forever,
but the chains looked fine. (The results were NOT fine, but that's
another story). Thinning is generally not encouraged any longer as a
method of reducing autocorrelation, but with such a large number of
stochastic nodes you might have to thin a bit just to save the samples
to disk. Don't forget that DIC is relative, not absolute, so I
wouldn't worry too much about its value. To reiterate Thomas' point,
the important thing is that your parameters (especially the
subject-level ones here) have converged.

[Frances Jin]

Thank you so much Sam for the detailed reply! My follow-up question is what you recommend to examine the subject-level convergence? I mean I do check the chains and auto-correlation by plotting the posterior_plot and do PPC of my best-fitting model. What else would you recommend?

Thank you in advance, again!

On Wednesday, April 24, 2019 at 5:48:23 PM UTC-4, Sam Mathias wrote:

I once used HDDM on a data set with ~2,000 subjects. It took forever,
but the chains looked fine. (The results were NOT fine, but that's
another story). Thinning is generally not encouraged any longer as a
method of reducing autocorrelation, but with such a large number of
stochastic nodes you might have to thin a bit just to save the samples
to disk. Don't forget that DIC is relative, not absolute, so I
wouldn't worry too much about its value. To reiterate Thomas' point,
the important thing is that your parameters (especially the
subject-level ones here) have converged.

On Wed, 24 Apr 2019 at 08:19, Thomas Wiecki <thomas...@gmail.com> wrote:
>
> That's fine, as long as your chains converged.
>
> On Wed, Apr 24, 2019, 14:15 Frances Jin <jinjin...@gmail.com> wrote:
>>
>> Hi HDDM experts,
>>
>> I am fitting models using more than 500 subjects, with each having more than 200 trials (but imbalanced across conditions). My first try using 5000 samples, 500 burn in and 5 thin resulted in a large Negative DIC.
>> I understand that DIC can be negative, but I am not sure if I have used the best model set up, particularly given my large sample size.
>>
>> Could someone let me know if the negative DIC is expected? Also, is my sampling, burn in, and thinning a reasonable choice? Any other things I need to pay attention to with large data?
>>
>> Many thanks, in advance!
>>
>> Best,
>> Frances
>>
>> --
>> You received this message because you are subscribed to the Google Groups "hddm-users" group.

>> To unsubscribe from this group and stop receiving emails from it, send an email to hddm-...@googlegroups.com.

>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups "hddm-users" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to hddm-...@googlegroups.com.

[Sam Mathias]

I think those are the main things to check.

> To unsubscribe from this group and stop receiving emails from it, send an email to hddm-users+...@googlegroups.com.

[Frances Jin]

Thank you very much, again!