bayesian anova on estimated parameters

[Olegna]

Hi all,

I have a complex design and I have fitted an unconstrained model, hence I end up having a lot of parameters. Rather than doing pairwise hypothesis testing on the estimated parameters (which doesn't seem the right thing to do to me since I'll end up with dozens of comparisons), I am thinking to enter the parameters into a repeated measures bayesian ANOVA and base my conclusions on its results.

Is it appropriate to do so or am I violating any assumptions I am not aware of (I know it is wrong to enter subject parameters in a frequentist ANOVA but is it the same for a bayesian anova)?

Thanks :)

[Samuel Mathias]

What exactly do you mean by "unconstrained"?

[Olegna]

Hi Samuel,

unconstrained means that boundary, non decision time and drift are free to vary across all conditions....hence there is no constrain for these parameters to be constant for some specific conditions.

[Samuel Mathias]

I that case, I believe its fine to extract your estimated parameters per subject and condition and use them as dependent variables in another kind of analysis, such as a Bayesian ANOVA (or frequentist ANOVA, for that matter). This kinda defeats the purpose of HDDM, but I don't see anything wrong with it.

[Mads Lund Pedersen]

So, in the ANOVA you will be doing comparisons between conditions that were already separated in the HDDM-analysis. The way I see it this could potentially be a problem. Don’t know the technical term for this, or if there is one, but you could imagine that subject-parameters for each condition will be estimated closer to the group mean for that condition (shrinkage). If you then do another hierarchical bayesian (ANOVA) model, the extracted subject-parameter values from the HDDM will again be drawn towards the mean of the group, and you can therefore end up with bigger condition-differences in the ANOVA. 

On the other hand, doing pairwise comparisons on group parameters in the HDDM should be totally ok, at least according to my understanding of bayesian data analysis. 

--
You received this message because you are subscribed to the Google Groups "hddm-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hddm-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Samuel Mathias]

Ah yes, that's correct, you would get shrinkage along the same dimensions that you would subsequently model in the ANOVA, so you would end up with bigger differences between levels of your factors (or more accurately, smaller variance within within levels). Sorry, I don't know what I was thinking just then. Perhaps I was imagining a situation where you were analyzing data from an experiment with a between-subjects design, and "unconstrained" meant on DDM parameter per subject, so that shrinkage would occur not across the dimensions of interest (i.e., the different groups in the experiment) but rather to the group as a whole. This or something similar came up in a previous question on the mailing list.

I agree, comparing posterior parameters is definitely the preferred route for HDDM. I guess Olegna's logic for preferring to perform an ANOVA is that one could just look at the main effect of a factor, which might have multiple levels, rather than comparing all levels to each other, hence cut down on the number of "comparisons". If so, I don't entirely agree. If you go down the ANOVA route, find a main effect, then perform post-hoc contrasts between levels to find the source of the effect, you've ended up doing the same number of comparisons anyway. If you were worried about spuriously concluding that a given condition influenced a particular DDM parameter you could fit two DDMs to the data, one where the parameter varies as a function of the condition and one where it doesn't, and see which model is a better fit.

[Olegna]

Ok thanks both for the detailed answer...I'll stick to the posterior parameters comparison then or do model comparison.

Thanks!