Re: accuracy and stimulus coding

[Thomas Wiecki]

Hi Jerome (cc'ing hddm mailing list),

On Fri, Oct 19, 2012 at 7:57 AM, Jerome Sackur <jerome...@gmail.com> wrote:

Dear Thomas,

I've started doing some analyses with hddm, and I stumble on a very basic and certainly very naive question about accuracy and stimulus coding: as I understand it, accuracy coding assumes that there is no bias, right? Then the drift rate will be positive whenever accuracy is above .5 and negative otherwise. But it makes no sense in this case to estimate a bias.

This is what I understand from the topic "Code subject responses" where you write:

"There are two ways you can code subject responses (these are the values you put in the ‘response’ column in your data file). You can either use accuracy-coding where 1 means correct and 0 means error, or you can use direction-coding where 1 means left and 0 means right (this could also code for stimulus A and B instead of left and right). HDDM interprets 0 and 1 responses as lower and upper boundary responses, respectively, so it has no preference one way or another."

But then you go on saying that sometimes, of course, we may want to include a bias. Thus I infer that in accuracy coding, you do not try estimate a bias (meaning that it is fixed at .5*a or 0). And this is also what my intuition tells me: it makes no sense to have a bias leading to being correct!

Yes, that's exactly correct. What you can do alternatively though, is not use accuracy coding but stimulus or direction coding and then estimate bias because you can be biased to respond left, or to respond "face".

HTH,

Thomas

 

All the best,

Thanks for making this great package available!

J.

--
Jérôme Sackur

Laboratoire de Sciences Cognitives et Psycholinguistique (ENS/CNRS/EHESS)
Département d'Etudes Cognitives
Ecole Normale Supérieure
29, rue d'Ulm --- Pavillon Jardin
75005 Paris, France
tel.: + 33 (0) 1 44 32 26 25
fax: + 33 (0) 1 44 32 26 30
email: jerome...@gmail.com

[Jerome Sackur]

Thanks! but when I do stimulus coding, I must input the stimulus as an independent variable (one more column in the data set) and make the drift depend on it. But suppose I have no reason to believe that the absolute value of the drift rate should be different in the two conditions, is there a way I can enforce v_red = - v_green (in case I have two targets, one red and one green)? If I don't do that, I will run into overfitting issues...

J.

[Thomas Wiecki]

On Fri, Oct 19, 2012 at 9:04 AM, Jerome Sackur <jerome...@gmail.com> wrote:

Thanks! but when I do stimulus coding, I must input the stimulus as an independent variable (one more column in the data set) and make the drift depend on it. But suppose I have no reason to believe that the absolute value of the drift rate should be different in the two conditions, is there a way I can enforce v_red = - v_green (in case I have two targets, one red and one green)? If I don't do that, I will run into overfitting issues...

Right, there has been a lengthy discussion on this here:

https://groups.google.com/forum/?fromgroups=#!topic/hddm-users/CV17JiSp0ao

Currently you can't enforce this, but this thread discusses a possible way to code this up. I'm not sure if Guido actually implemented that or what the status of that is.

[guido biele]

hi,

it is implemented and it works (thanks to Thomas patient explanations :-)!. I wanted to test it in a larger data set before posting it to the list, but we got a new cluster and installation of the required modules was delayed.

anyhow, I'll post the current method later tonight (European time)

guido

--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

[Guido Biele]

Hi,

attached are 2 scripts.

test_reduced.py is and example for how to run the overloaded hddm.
InvZ.py is that script the loads hddm and created the overloaded method.

I don't typically use python, so please excuse if the code isn't very
clean (I don't) think it is to bad

Cheers - Guido

> <tel:%2B%2033%20%280%29%201%2044%2032%2026%2025>

> fax: + 33 (0) 1 44 32 26 30

> <tel:%2B%2033%20%280%29%201%2044%2032%2026%2030>
> email: jerome...@gmail.com
> <mailto:jerome...@gmail.com>

>
>
>
>
>
>
> --
> Jérôme Sackur
>
> Laboratoire de Sciences Cognitives et Psycholinguistique
> (ENS/CNRS/EHESS)
> Département d'Etudes Cognitives
> Ecole Normale Supérieure
> 29, rue d'Ulm --- Pavillon Jardin
> 75005 Paris, France
> tel.: + 33 (0) 1 44 32 26 25

> <tel:%2B%2033%20%280%29%201%2044%2032%2026%2025>

> fax: + 33 (0) 1 44 32 26 30

> <tel:%2B%2033%20%280%29%201%2044%2032%2026%2030>
> email: jerome...@gmail.com <mailto:jerome...@gmail.com>
>
>
>

[Jerome Sackur]

Thanks a lot for all that!

I'll look into it and tell you how it worked out,

Best,

Jérôme

[Thomas Wiecki]

Hi Guido,

Good to see that it worked. Code seems pretty straight forward, thanks for the contribution!. A couple of notes:

- I don't think you need all of those imports

- You probably don't have to overload create_knodes(). It's identical from what I can tell.

- If you inherit from HDDM instead of HDDMBase you will get the new model with transformations and Gibbs sampling.

I did those and some other changes and included the model in HDDM; the develop branch now has a new model called HDDMStimCoding with some documentation. I also updated the howto with some pointers on how to use this new model:

http://ski.clps.brown.edu/hddm_docs/howto.html#code-subject-responses

Let me know if there are any problems with this.

Thomas

<mailto:jerome.sackur@gmail.com>

--
Jérôme Sackur

Laboratoire de Sciences Cognitives et Psycholinguistique
(ENS/CNRS/EHESS)
Département d'Etudes Cognitives
Ecole Normale Supérieure
29, rue d'Ulm --- Pavillon Jardin
75005 Paris, France
tel.: + 33 (0) 1 44 32 26 25

<tel:%2B%2033%20%280%29%201%2044%2032%2026%2025>

fax: + 33 (0) 1 44 32 26 30

<tel:%2B%2033%20%280%29%201%2044%2032%2026%2030>

email: jerome...@gmail.com <mailto:jerome.sackur@gmail.com>

[guido biele]

hi thomS ,
thanks!
about create_knodes: I think I got an error message when I didn't include it. but maybe I was missing something else which caused that problem.

cheers-guido

PS: [very technical] do you have any experince with compiling pymc with the Intel MKL library? this should make a considerable difference on Intel processors (compared to atlas libraries) but due to lacking the experience I can't seem to find a way to point the setup to the Intel BLAS and LAPACK libraries...