Complicated within-subjects design (2x2x7)

[Marty]

Hi all,

First, the HDDM package is great.  I've found, however, that it works best with relatively simple designs, particularly for bivariate cases (condition A vs B, or group A vs B).  I've been spending some time now trying to apply this to a more complicated, within-subjects design (2x2x7).  I've come up with my strategy, and this is what I want to present, but I just want to make sure that folks will be OK with this.

1) I started out fitting my data with HDDMStimCoding.  Data were hierarchical, everything looks good.  The model is complicated, it includes v, a, t, and z as model parameters.  I have a-priori reasons for this, as I'm trying to replicate an earlier DDM paper (that didn't use HDDM).  In the end, everything looks great - PPC is good, MCerr is good, R-hat is good, visual inspection of chains is good.  But, how to compare the conditions?  Simple bivariate comparison of posteriors won't work, because I'm looking for a complex interaction across these factors.  I can run a frequentist ANOVA, and the effect is there, I can run a Bayesian ANOVA (in JASP) and the effect is there.  But, isn't this a violation?  Bayesian ANOVA still has an assumption of independence, I believe.

2) Ok, so what about non-hierarchical fitting?  In this case, I went and fit each subject individually, using HDDMStimCoding.  I then append my dataframe with the results and look at the findings.  Individual parameters look good, but MCerr is a little odd for random subjects and random parameters.  BUT, the same effects are there as the hierarchical, and values (on average) are around the same.  Again, run the ANOVAs, and the effects are there.

3) So, fine, why not use HDDMRegressor?  Happy to do so.  I go ahead and run it, and again everything converges and looks nice.  I examine my posteriors, and find that the regression coefficient for the interaction I'm interested in is greater than zero (more than 95%).  Fantastic, but this doesn't quite capture the pattern that I want to show (working with intercepts and coefficients in this design works, but I want to show my results side-by-side with a specific previous paper).  

In sum, I have three separate analyses that all converge on the same particular effect.  If I want to present this in a paper, is it alright if I present all three of these analyses?  Or, should I throw out the HDDMStimCoding analysis in favor of HDDMRegressor?  I'm reluctant to do so, given that the HDDMStimCoding analysis nicely presents the interaction effect, which is very interpretable in light of this earlier paper.  I want to make sure I'm doing the right thing, here.  (I'll note that I'm also using HDDM for some other, cleaner designs, in which the analysis and presentation of results is much more straightforward, it's just this dang complicated 2x2x7...).

Apologies for the long summary, I just want to get some consensus.