Scoring of multiple answer items

[Daniel Sommerhoff]

Dear Phil, dear Community,

I'm currently trying to implement multiple choice items into mirtCat. In principle, everything works fine. I was able to load a csv-file with checkbox-questions and multiple options and multiple answers for each question. mirtCAT works just fine that far.

However, the scoring mirtCAT uses rewards every correct answer with one point, so that a question with three correct answers can give up to three point. At the same time checking wrong answers does not impact the score of the item.

What I would like to do is to: "Score an item with 1 point if and only if all correct answers are checked."

Example:

Which of the following are prime numbers:

Answer options: a)1 b)2 c)3 d)4 e)5

Correct answers: 2,3,5

Possible answer by a student #1: 1,2,3,5 -->results in 0 points

Possible answer by a student #2: 2,3,5 -->results in 1 points

Possible answer by a student #3: 2,3 -->results in 0 points

Is there any way to change the internal scoring of mirtCAT?

Thanks in advance for your help, it's really appreciated.

Best,

Daniel

[Phil Chalmers]

Hi Daniel,

The argument AnswerFuns is exactly what you want here. The purpose of this list input is to specify how the observed responses are translated into suitable scoring forms for each respective item. Here's a trivial example reflecting how this works:

library(mirtCAT)

options <- matrix(c("Strongly Disagree", "Disagree", "Neutral", "Agree", "Strongly Agree"),

                  nrow = 3, ncol = 5, byrow = TRUE)

questions <- c("Building CATs with mirtCAT is difficult.",

               "mirtCAT requires a substantial amount of coding.",

               "I would use mirtCAT in my research.")

df <- data.frame(Question = questions, Option=options, Type = 'checkbox')

AnswerFuns <- as.list(rep(NA, nrow(df)))

AnswerFuns[[1]] <- function(text) all(text %in% c("Strongly Disagree", "Disagree"))

AnswerFuns[[2]] <- function(text) all(text == "Neutral")

AnswerFuns[[3]] <- function(text) all(text == "Strongly Agree")

results <- mirtCAT(df = df, AnswerFuns=AnswerFuns)

summary(results)

The first question will only be given a 1 if the first two categories (and only the first two categories) are selected;  otherwise, it will give a 0 (note that TRUE and FALSE evaluate to 1 and 0 in R, respectively. Though you can use this idea to return other numbers as well). The next two functions really aren't needed since they are supported within the df object directly, but demonstrate how this same line of reasoning can be applied to other items in the pool. HTH, and let me know if you have any questions.  

 

Phil

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Daniel Sommerhoff]

Great, thanks a lot! Worked like a charm :-) Sorry for not seeing it myself...

[Daniel Sommerhoff]

Hi Phil,

just as a short addendum to your answer to my initial question:

AnswerFuns #1

function(text) all(text %in% c("Strongly Disagree", "Disagree"))

This function checks if the answers provided by the participants are elements of "Strongly Disagree", "Disagree". It will thus return TRUE as long as participants only check correct answers. Yet, it will also return TRUE is participants don't check all correct answers.

AnswerFuns #2

function(text) setequal(text,c("Strongly Disagree", "Disagree"))

This function checks if the answers provided by the participants equal the correct answers "Strongly Disagree", "Disagree". It will thus return TRUE only if the participants check all correct answers.

AnswerFuns #1 thus does not fully fulfill the intended scoring asked for in my initial post ( "Score an item with 1 point if and only if all correct answers are checked.") and I wanted to clarify this so that future readers don't make a mistake here.

Thanks again @Phil.

Daniel

[Phil Chalmers]

On Thu, Aug 23, 2018 at 5:34 AM Daniel Sommerhoff <daniel.s...@gmail.com> wrote:

Hi Phil,

just as a short addendum to your answer to my initial question:

AnswerFuns #1

function(text) all(text %in% c("Strongly Disagree", "Disagree"))

This function checks if the answers provided by the participants are elements of "Strongly Disagree", "Disagree". It will thus return TRUE as long as participants only check correct answers. Yet, it will also return TRUE is participants don't check all correct answers.

Thanks. I think it's because the order is backwords. Compare  the above function with 

all(c("Strongly Disagree", "Disagree") %in% text)

But then, if the input is something like a checkbox this would return TRUE when text has more than two elements. 

AnswerFuns #2

function(text) setequal(text,c("Strongly Disagree", "Disagree"))

This function checks if the answers provided by the participants equal the correct answers "Strongly Disagree", "Disagree". It will thus return TRUE only if the participants check all correct answers.

I like this form. Thanks for sharing.

Phil 

AnswerFuns #1 thus does not fully fulfill the intended scoring asked for in my initial post ( "Score an item with 1 point if and only if all correct answers are checked.") and I wanted to clarify this so that future readers don't make a mistake here.

Thanks again @Phil.

Daniel

On Wednesday, August 1, 2018 at 3:50:47 PM UTC+2, Daniel Sommerhoff wrote:

Great, thanks a lot! Worked like a charm :-) Sorry for not seeing it myself...

--