A few questions on within-subjects design

[BIlly Zhu]

Hello everyone, I'm new to the HDDM package, so please excuse my questions if they seem less specialized. I have two main inquiries; the first concerns the format of the variables, and the second pertains to the differences in result output between hddm.HDDM() and hddm.HDDMRegressor().

Experimental Background:
My experiment involves a combination of one between-subject variable (A, B) with two within-group variables. The first within-subject variable is "magnitude" (none, small, large), while the other is "valence" (reward, punishment). When recording the data, half of the "none" instances corresponded to rewards, and the other half to punishments. However, in reality, the subjects were exposed to the same stimuli.

Therefore, in the "none" condition, both reward and punishment should yield the same reaction times (RTs) and accuracy.

Initially, my thoughts were as follows:

model = hddm.HDDM(data, include=['v','z','a','t'],
                  depends_on={'v':['type','magnitude','valence'],
                              'a':['type','magnitude','valence']})

However, this approach would treat instances when "magnitude" is "none" separately for reward and punishment conditions, causing differences in the treatment of the same stimulus.

Another idea would be to consolidate the two variables into a single variable "Stimulus," with levels such as (punish_small, punish_large, neutral, reward_small, reward_large):

model_c = hddm.HDDM(data, include=['v','z','a','t'],
                    depends_on={'v':['type','stimulus'],
                                'a':['type','stimulus']})

May I ask if this approach is feasible? If so, is there any relevant literature that supports this method?

Second question:
Given that this is a mixed design, it appears that instead of using hddm.HDDM() with "depends_on," it might be more suitable to use hddm.HDDMRegressor(). However, my primary reason for not using hddm.HDDMRegressor() is that, the output does not provide parameter values for each subject under different conditions.

For instance, by setting depends_on={'v':['type','magnitude','valence']} in hddm.HDDM(), the output displays parameter values (v/a) for each condition and each subject within the combination of magnitude and valence.

However, with hddm.HDDMRegressor(), the output does not report parameters for each subject in each condition; it only shows the parameter for that subject throughout the entire experiment.

As with the first question, how should I construct my variables if I choose to use hddm.HDDMRegressor? Should I use "magnitude" and "valence" separately or combine them as "stimulus"?

Thank you in advance for your guidance and assistance.

Billy

[Guillaume Pagnier]

Hi Billy,

HDDM does not support mixed effects (HSSM does though if you want to switch to that). That being said you can still investigate the questions of interest with HDDM. My understanding of your task is that an initial approach could be to fit two separate depends_on models for populations A and B and then report the posteriors from both models (this assumes you have enough participants in both populations). I would combine both predictors into a 'stimulus' and just use HDDM depends_on (once again, assuming you have enough trials within each stimulus group and that you have no reason to believe trials within a stimulus group would differ from each other). HDDMRegressor while powerful, comes at the cost of assuming a constant relationship between the model parameter and covariate. It's especially useful if you have trial by trial differences but that doesn't seem to be the case in your design and what you really care about in any case is how parameters change across those five stimuli (punish_small, punish_large, neutral, reward_small, reward_large). For the record, HDDMRegressor can report individual model estimates, you just have to make sure to set group_only_regressors to False. 

Hope this is helpful.

~Guillaume

[BIlly Zhu]

Hi Guillaume

Thank you very much for your reply.
For your advice it has helped me to sort out my problem and it has helped me a lot.

I have a new query is it convenient for you to answer? 

There are some trials where the participant did not response (missing).

So in the data the response would be recorded as 0 and the RT would be NAN.

In this case it seems that need to drop the trials where the RT is NAN before hddm.utils.flip_errors(data) and model.sample, instead of adding the p_outlier=0.05 as such in the hddm.HDDM(data, p_outlier=0.05).
Because it seems that I would get an error if I did not remove the NAN trials and just do the model.sample with p_outlier=0.05.

Therefore, when processing the data, is how should I treat the trials where the participant did not the response at all?

Once again, I would like to express my gratitude to you!

Billy

[BIlly Zhu]

Hi Guillaume,

Thanks again for your help!

Based on my understanding of your suggestion, I have made changes to the code.

(cite: fit two separate depends_on models for populations A and B and then report the posteriors from both models)

Code based on your suggestion 

model_c_A = hddm.HDDM(data[data[type=='A']], include=['v','z','a','t'],
                    depends_on={'v':'stimulus',
                                             'a':'stimulus'})

model_c_B = hddm.HDDM(data[data[type=='B']], include=['v','z','a','t'],
                    depends_on={'v':'stimulus',
                                             'a':'stimulus'})

Am I understanding you correctly? 

If yes, is there a difference between above code and the one below?

model_c = hddm.HDDM(data, include=['v','z','a','t'],
                    depends_on={'v':['type','stimulus'],
                                'a':['type','stimulus']})

Thank you very much for your advice!

Billy

在2023年10月11日星期三 UTC+8 02:43:05<guillaum...@brown.edu> 写道：

[Guillaume Pagnier]

Your first code (models c_A and c_B) is valid and more appropriate (here is an example of computational differences emerging from running models on separate populations). Your second code (model_c) is not the same as the first (HDDM won't assign type as a between-subject factor) and due to the hierarchical nature of HDDM, the posteriors for each 'type' will interfere with one another and lead to incorrect conclusions. As always, to validate your models, run appropriate posterior predictive checks to confirm each model is capturing the critical patterns of the data on its respective group. 

[BIlly Zhu]

Hi guillaume,

I am very sorry for getting back to you so late.

Your answers are very important to me and my questions were answered well thanks to your support.

Once again, I would like to express my gratitude.

Billy