Stim Coding with no correct answer

[Martin Wiener]

Hi everyone,

I've been using the HDDM toolbox for running some analyses for a few months now (it's really great), and have been toying around with a few paradigms. 

One question I have in particular is when using the HDDMStimCoding module.  In this case, the documentation and website state that a the stim_col column needs to contain a vector indicating what the *actual* correct response was.  However, what if there is no correct response?  I have a particular task that I would like to apply the model to, but in it the subjects are left to decide between two alternatives, yet there one condition where the stimulus is completely ambiguous, and so subjects must guess one way or another.  Is there a way that the StimCoding module can be used, or the data can be coded, such that the accuracy information is unnecessary?  I may be misunderstanding the explanation in the documentation though, so I just wanted to check before going ahead. 

~Martin Wiener

[Thomas Wiecki]

Hi Martin,

In that case I think you can just use the regular HDDM and use stimulus coding.

Thomas

--
You received this message because you are subscribed to the Google Groups "hddm-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hddm-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Thomas Wiecki, PhD

Data Science Lead, Quantopian Inc, Boston

[Martin Wiener]

Ok, thanks!  This is what I suspected, but I'm glad to see it confirmed.

~Martin

[Martin Wiener]

I have a follow up question:

I tried running my model (estimating bias) in both HDDM and HDDMStimCoding, just to compare the results.  Everything lines up with regards to other parameters (drift, threshold, non-decision-time), but I get different results for bias (z).  I expected this, because HDDM and HDDMStimCoding approach bias in different ways (as I understand it).  However, I just want to make sure that my interpretation is correct:

Bias is normally fixed at 0.5, and should range between 0 and 1.  If I estimate z in HDDM, does a high value of z mean that the subject is more biased to the incorrect boundary, and a low value means they are biased to the correct boundary?  But, if I estimate z in HDDMStimCoding, does a high value of z mean a bias to the upper boundary (say, coded as response choice 1), and a low value mean a bias to the lower boundary (response choice 0).  Is this correct?  If not, then could it be explained what the difference in bias between HDDM and HDDMStimCoding is?  I can provide more details if it's needed for a clearer answer, my apologies if this is a simple question!

~Martin  

[Sam Mathias]

My understanding was that in regular “accuracy” coding the lower boundary is set at zero, such that the parameter a represents both the absolute value of the upper boundary and the distance between the boundaries. In this statement is true, than z > 0.5 means bias towards the CORRECT boundary, not the incorrect boundary. Please note I haven’t the source code to verify this.

Samuel R. Mathias

Postdoctoral Associate

Department of Psychiatry, Yale School of Medicine

Suite 3014, 2 Church Street South

New Haven, CT 06519

http://www.srmathias.com

[Thomas Wiecki]

That's correct, Sam.

It might be informative to note that HDDM has no concept of correct or incorrect. Only upper and lower boundary.  How you interpret and code responses is up to you. 

[Juan]

Hi, list experts,

I just check this post, as I realize that I happened to meet a similar problem. I quickly checked my data. Firstly, my behavioral data show that RT(reward) < RT(threat) < RT(neutral) and all accuracies were very high (mean: 96%).

I tried both hddm.HDDM and hddm.HDDMRegressor methods for my data - I only free the parameter z. The outputs were very close, but all Z<0.5 (Zreward =.35, Zthreat =.30, Zneutral =.26), which means that the bias pointed to the incorrect boundary(?). Clearly they must be wrong.  On the other hand, it seems that have no reason to use hddm.HDDMStimCoding, as the accuracy-coding is the same to stimulus-coding in my data set. I am puzzled on this, as I do not find any problem with my data analysis (scripts). Do you guys always check the Z values before you do the final comparisons?

Any comments are much appreciated,

J

[Thomas Wiecki]

That still makes sense in that higher Z should lead to faster correct responses. It does suggest that you have fast errors in your data though. What's the mean RT of correct and error trials separately in these 3 conditions?

[Juan]

Hi, yes, I get it. thomas -  The Z<.5 indicates a long RT for the contrary/incorrect boundary

I have now checked and found that RT(correct) is longer, compared to the corresponding RT(incorrect). Also, there are speed-accuracy tradeoff appeared with my correct behavioral data. 

Thanks so much.

[Juan]

A further question, has anyone  tried to examine the proportion of the posteriors in which the DDM parameters (e.g., v) for one condition is greater than the other, using this stimulus coding HDDMRegression analysis?

After I finished the modeling, I tried the following method (copied from the demo), but they did not work. It simply gave me an error alarm: AttributeError: 'float' object has no attribute 'trace'.

m_reg = hddm.HDDMRegressor(mydata, reg_descr, include='z')

v_WL, v_LL, v_WW = m_reg.nodes_db.ix[["v_Intercept",
                                              "v_C(stim, Treatment('WL'))[T.LL]",
                                              "v_C(stim, Treatment('WL'))[T.WW]"], 'node']


print "P(WW > LL) = ", (v_WW.trace() > v_LL.trace()).mean()
print "P(LL > WL) = ", (v_LL.trace() > v_WL.trace()).mean()

I guess that one reason is that the program could not know which should be treated as intercept(?). When we do the common within-subject modeling, we first run a dmatrix step, and claim which condition would be used as intercept, like

dmatrix("C(stim, Treatment('WL'))")

Any good pointers? Thanks a lot in advance. 

[Thomas Wiecki]

Hi Juan,

can you investigate closer what v_WW.trace is? Just printing it, getting it's type etc. That would help in figuring this out. Perhaps v_WW.trace[:] works?

Thomas

[Chris Street]

Hi,

Sorry to jump into the middle of a thread, but I was looking for some clarification on some of the replies offered so far.

On Wednesday, June 24, 2015 at 4:10:43 AM UTC-4, Thomas wrote:

That's correct, Sam.

It might be informative to note that HDDM has no concept of correct or incorrect. Only upper and lower boundary.  How you interpret and code responses is up to you. 

On Thursday, June 18, 2015 at 5:11:51 PM UTC-4, Thomas wrote:

Hi Martin,

In that case I think you can just use the regular HDDM and use stimulus coding.

Thomas

On Thu, Jun 18, 2015 at 8:14 PM, Martin Wiener <martin.d...@gmail.com> wrote:

Hi everyone,

I've been using the HDDM toolbox for running some analyses for a few months now (it's really great), and have been toying around with a few paradigms. 

One question I have in particular is when using the HDDMStimCoding module.  In this case, the documentation and website state that a the stim_col column needs to contain a vector indicating what the *actual* correct response was.  However, what if there is no correct response?  I have a particular task that I would like to apply the model to, but in it the subjects are left to decide between two alternatives, yet there one condition where the stimulus is completely ambiguous, and so subjects must guess one way or another.  Is there a way that the StimCoding module can be used, or the data can be coded, such that the accuracy information is unnecessary?  I may be misunderstanding the explanation in the documentation though, so I just wanted to check before going ahead. 

~Martin Wiener

--
You received this message because you are subscribed to the Google Groups "hddm-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hddm-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

If I understand correctly, HDDM only knows that there are two decision boundaries and makes no assumptions about the incoming data (e.g., that accuracy is coded, and so accurate judgments will be modelled as being faster). All it gets are a set of 0s and 1s with no assumption about what they represent and it's down to us to interpret the parameters correctly. Have I understood that correctly?

My other question is the reply to Martin's original question, which said he could use regular HDDM and use stimulus coding even though there is no accuracy column. Does that mean that the first input to hddm.HDDMStimCoding is mandatory but all the following input arguments are optional? And could the below be written without concern?

model = hddm.HDDMStimCoding(data, include='z', split_param='v')

I'm very new to HDDM, sorry for the naive question. Thanks for your help in advance,

Chris

[Sabina Nowakowska]

Hello, 

I work on a similar problem as Martin (multiple conditions and one of them is ambiguous with no correct answer) and I have a similar question as Chris. I was wondering, what is actually a difference between the standard HDDM model and HDDMStimCoding, since the algorithm only knows upper and lower boundary not the meaning of it? Using hddm.HDDM where response column has information left-right instead of correct-incorrect should yield the same result as using HDDMStimCoding, right?

best,

Sabina