error penalties

[Melissa Diliberti]

Hello,

My colleagues and I are using the IAT code (specifically the IAT8) from MinnoJS to integrate the IAT into our survey. 

It is our understanding that in scoring the IAT respondents should either (1) not be allowed to move forward until they have corrected an error that they have made [in which case the "penalty" for their error is built into their latency for that trial] OR (2) be allowed to move forward when they make a mistake without correcting it, but instead be tagged with a 600ms error penalty.

In looking at the IAT8 code on MinnoJS, however, it looks like the default setting is to have respondents have to correct their response and also get tagged with the 600 penalty (which seems to contradict the required scoring practices). Specifically, in lines 147 and 1154

147: errorCorrection : true, 

1154: errorLatency : {use:errorLatencyUse, penalty:600, useForSTD:true},

Our questions are:

(1) Can you confirm that our interpretation of the code is correct (i.e., that in default settings, users are getting penalized twice by having to correct their response but also get the 600ms hit)?

(2) Relatedly, that we should turn one of these settings "off" to avoid double penalizing respondents?

(3) Can you confirm that these time penalities are already reflected in respondents' latencies when we download the data?

Thank you for your time and assistance. We really appreciate it.

All the best,

Melissa 

[Mayan Navon]

Hi Melissa,

First, the score you get in the csv output SHOULD NOT be used for data analysis. It is simply meant to be used to present participants with feedback on their IAT performance. The raw data you get in the csv output does not use any algorithm. The code you are referring to computes the score for the feedback variable. So the answer to your last question is 'no'.

Regarding your interpretation of the code, in the case that errorCorrection is true, errorLatencyUse will be equal to 'latency', rather than 'penalty'.

I hope that helps.

Best,

Mayan

--
You received this message because you are subscribed to the Google Groups "Minno.js" group.
To unsubscribe from this group and stop receiving emails from it, send an email to minnojs+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/minnojs/233b3fd7-e8fb-4eba-aad7-2fa9b4c902a3n%40googlegroups.com.

--

Mayan

[Mayan Navon]

Hi Melissa,

Yes, the latency is for the time to get to the correct response. This is why we don't need to add a penalty in our scoring algorithm, when using this task procedure.

Yes, 0 = correct, 1 = incorrect.

Best,

Mayan

On Mon, 25 Oct 2021 at 23:17, Melissa Diliberti <mkdil...@gmail.com> wrote:

Hi Mayan,

Thanks for the prompt reply; we appreciate it!

I understand from your response that we should not use the auto-generated scores and should instead create the D-scores ourselves using the raw output. This is fine, and we will do so. 

Just to confirm our understanding of the raw data: for trials where the respondent initially answers incorrectly and has to change their response to be correct before moving forward, the latency is the time to get to the correct response, not the time to provide the initial (incorrect) response. Is this correct?

We are also interpreting 0s to be instances where the respondent gets the trial correct, and 1s as cases where the respondent initially makes an incorrect response. Can you confirm this is a correct interpretation?

Really appreciate your help! All the best,

Melissa

--

Mayan