how to estimate a (future) score when item response pattern is not represented in fscores table

[Steven Vannoy]

Hi,

I'm new to IRT and have not been able to figure out the answer to this question despite several hours of reading the manual and websites. I know it  is a fundamental ignorance on my part so any help is much appreciated.

I have a 9 item Likert-type scale and am considering these ordinal items. I've administered the scale to a large number (> 1000) of participants. The scale is commonly used in my field by just calculating scores as item sums. I'm curious about how IRT might enhance the use of this scale, there is definitely variance in the difficulty of items.

My understanding is that I can use fscores to get a table that provides a score "F1" for each item response pattern that was in the data set. So, if I wanted to get a score for a new respondent, I could match their response pattern with the fscores table and get an estimated score. But, in my original data set, not all response patterns are represented. My question then, is, can I calculate an (estimated) score from the coefficients (or some other way) for a response pattern that was not present in the original analysis? I realize that this estimated score might have a large standard error as a result, but I can live with that, I'm mostly wanting to understand how this would be done procedurally.

I suppose one way is to simply add that respondent to my dataset and reanalyze. But I feel like I'm missing something fundamental here about the information provided by an IRT analysis.

Just for some specific data, here is some output from my analysis:

mod_gpcm <- mirt(itemDf, 1, itemtype = 'gpcm', verbose = FALSE)

> coef(mod_gpcm, simplify=TRUE)

$items

        a1 ak0 ak1 ak2 ak3 d0     d1      d2      d3

itm1 2.133   0   1   2   3  0 -1.413  -4.609  -8.655

itm2 2.638   0   1   2   3  0 -2.536  -7.369 -12.187

itm3 1.131   0   1   2   3  0 -0.163  -1.097  -2.139

itm4 1.831   0   1   2   3  0  0.267  -1.640  -4.142

itm5 1.580   0   1   2   3  0 -1.339  -3.584  -6.247

itm6 1.665   0   1   2   3  0 -2.501  -5.536  -8.320

itm7 1.938   0   1   2   3  0 -1.590  -4.476  -7.913

itm8 1.920   0   1   2   3  0 -3.098  -7.467 -11.323

itm9 2.107   0   1   2   3  0 -5.176 -11.446 -17.713

> coef(mod_gpcm, simplify=TRUE, IRTpars = TRUE)

$items

         a     b1    b2    b3

itm1 2.133  0.663 1.498 1.897

itm2 2.638  0.961 1.832 1.826

itm3 1.131  0.144 0.826 0.922

itm4 1.831 -0.146 1.041 1.366

itm5 1.580  0.848 1.421 1.685

itm6 1.665  1.502 1.823 1.672

itm7 1.938  0.820 1.490 1.774

itm8 1.920  1.614 2.276 2.009

itm9 2.107  2.457 2.976 2.975

> head(fscores(mod_gpcm, full.scores = FALSE))

Method:  EAP

Empirical Reliability:

   F1 

0.791 

     itm1 itm2 itm3 itm4 itm5 itm6 itm7 itm8 itm9          F1     SE_F1

[1,]    0    0    0    0    0    0    0    0    0 -1.11835765 0.6541271

[2,]    0    0    0    0    0    0    0    1    0 -0.48372235 0.5046332

[3,]    0    0    0    0    0    0    1    0    0 -0.47914656 0.5035579

[4,]    0    0    0    0    0    0    2    0    0 -0.07728942 0.4132801

[5,]    0    0    0    0    0    0    2    2    0  0.42370274 0.3194128

[6,]    0    0    0    0    0    1    0    0    0 -0.55063076 0.5204260

Thank you for any help you can offer.

[Phil Chalmers]

Hi Steven,

Thanks for taking time to read the manual first (I must admit, they are getting a little large given all the arguments....). You were mostly there actually in your analysis. If you want to include future response patterns given a model that has already been fitted then fscores() has the input response.pattern = NULL, which can be replaced with any (valid) data in the future. 

library(mirt)

mod <- mirt(Science, 1)

fscores(mod, full.scores = FALSE)

newdata <- rbind(c(1,1,1,1), c(1,4,3,2))

fscores(mod, response.pattern = newdata)

HTH.

Phil

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Steven Vannoy]

Thanks for the super fast response. I must admit I feel better knowing that I really was pretty close. 

Can I ask, is it possible to obtain the scores from the coefficients? This is just to help my understanding in linking them together. Happy to go read a reference if you can point me to one that answers this question. I've been reading on-line tutorials and journal articles all over the place, and while it seems to me that the ultimate reason we want to use IRT is to estimate scores, that last step gets left off (I presume that most authors assume it is obvious and it probably is to most readers).

Sincerely,

Steven

Phil

To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.

[Phil Chalmers]

On Thu, Nov 30, 2017 at 5:55 PM, Steven Vannoy <steven...@gmail.com> wrote:

Thanks for the super fast response. I must admit I feel better knowing that I really was pretty close. 

Can I ask, is it possible to obtain the scores from the coefficients? This is just to help my understanding in linking them together. Happy to go read a reference if you can point me to one that answers this question.

I'm not sure what you mean by "from the coefficients"? Obtaining estimates generally requires iterative optimisation techniques or numerical integration, so simple formulae are not available. 

 

I've been reading on-line tutorials and journal articles all over the place, and while it seems to me that the ultimate reason we want to use IRT is to estimate scores, that last step gets left off (I presume that most authors assume it is obvious and it probably is to most readers).

That, and there generally are issues related to secondary scores (e..g, because they are computed by treating the IRT parameters as fixed and known....so if there's any inaccuracies in the estimates they get carried over. And for several other reasons).  

Phil

To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package+unsubscribe@googlegroups.com.