vcov-method From mirt v1.32.1 to extract Parameter Variance Covariance Matrix

[Monika Vaheoja]

Dear Phil,

I have a question regarding the variance covariance matrix that i can access by using vcov. Because the lack of statistical knowledge, i might ask a bit blond question. 

Im using Mirt (multigroup) for my research in which I: 

a. Simulate data with  simdata function for two exams and combine these to

b. Estimate parameters with multigroup

          spec = ('F=1-45,  CONSTRAINB = (1-45, a1)')

          spec_inv = c('free_means','free_var','intercepts')

          

          mirt.tmp <- multipleGroup(rf, model = spec, group = as.factor(df$bkl), invariance = spec_inv, SE=TRUE)

c. then I access parameters and covariance matrix

coef <- coef(mirt.tmp, simplify = TRUE, IRTpars = T)

cco <- vcov(mirt.tmp)[1:90,1:90]

And now my question: 

For my research im working with regular irt parameters and i need the variance-covariance matrix of the parameters, however, because in Mirt are the easiness parameters used, is it true that the covariance matrix is also for easiness parameters? of can i use it without any transformation? 

if i do need to transform these, what kind of way can i do it? 

Unfortunatelly i have found no literature about the exact computation of variance-covariance matrix in mirt. 

thank you !

Monika 

[Phil Chalmers]

Hi Monika,

The variance covariance in define with respect to the parameters defined, so yes it is parameterization sensitive (the ML parameter estimates are generally not parameterization sensitive though, which is why we can back-transform them into the more classical difficiculty-discrimination form). If you investigate literature pertaining to the observed (or expected) information matrix in the context of ML estimation you'll find a great deal about these mathematical structures. 

In your situation, if you want the VCOV for the classical parameterization you can do one of two things: estimate the models you actually want by creating them via createItem(), and obtain the vcov estimate given these user-defined models, or estimate some transformation of the vcov via approximations such as the delta method (this is what the coef(mod, IRTpars = TRUE, printSE = TRUE) output does). I suppose the bigger question would be, are you sure you need to use the original parameterization (i.e., what exactly do you need it for)? HTH.

Phil

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/56dbef66-023f-403d-bebe-b42e421ddd08n%40googlegroups.com.

[Monika Vaheoja]

Thank You Phil! 

I have the feeling that I need to transform the vcov, because I want to have classical parameters.  

'For what do i need all this trouble?' 

Im studying the standard error of expected scores in different contexts, such as the sample size, exams difficulty and population ability. In my research does the expected score represent a standard (a cut-score). 

We have different exam forms, on one of them ( the reference exam) experts set a standard as a test score (with Angoff method) and through IRT calibration (concurent calibration; thats the reason why i need multigroup) we compute an expected score on a new exam form given the theta value of the reference exam's score. However, to estimate the standard error of the expected scores on a new exam form I need covariance-variance matrix of the classical parameters to apply Delta method. 

Because we are comparing simulation results of different models (Rasch, One Paramater Logistic Model (Dutch Verhelst model) and 2PLM) I would like to keep all parameters the same, which means that I want to use classical parameters. And I need a vcov of the classical parameters. The reason why I use Mirt as an IRT package is that its the only IRT model estimation package that can handle incomplete design (correct me if i'm wrong). Ofcourse, I know, there are different ways to calibrate different exam forms, however, at this point I'm not willing to change the method for a dfferent calibration method. 

Do you have any suggestions to transform the vcov matrix to have the variance-covariance matrix of the classical parameters? And are you sure you are not willing to expand your package in which classical parameters will be used instead?

geetings, 

Monika    

[Phil Chalmers]

On Wed, Sep 30, 2020 at 2:53 AM Monika Vaheoja <monika...@gmail.com> wrote:

Thank You Phil! 

I have the feeling that I need to transform the vcov, because I want to have classical parameters.  

'For what do i need all this trouble?' 

Im studying the standard error of expected scores in different contexts, such as the sample size, exams difficulty and population ability. In my research does the expected score represent a standard (a cut-score). 

We have different exam forms, on one of them ( the reference exam) experts set a standard as a test score (with Angoff method) and through IRT calibration (concurent calibration; thats the reason why i need multigroup) we compute an expected score on a new exam form given the theta value of the reference exam's score. However, to estimate the standard error of the expected scores on a new exam form I need covariance-variance matrix of the classical parameters to apply Delta method. 

You can also find the expected standard errors from the slope-intercept form too; the math wouldn't be very different at all, and in fact you would likely get more optimal estimates since the slope-intercept form has a smoother likelihood shape than the classical parameterization (which actually has inconsistencies, such as when the discrimination gets too close to 0). 

 

Because we are comparing simulation results of different models (Rasch, One Paramater Logistic Model (Dutch Verhelst model) and 2PLM) I would like to keep all parameters the same, which means that I want to use classical parameters. And I need a vcov of the classical parameters. The reason why I use Mirt as an IRT package is that its the only IRT model estimation package that can handle incomplete design (correct me if i'm wrong). Ofcourse, I know, there are different ways to calibrate different exam forms, however, at this point I'm not willing to change the method for a dfferent calibration method. 

Do you have any suggestions to transform the vcov matrix to have the variance-covariance matrix of the classical parameters? And are you sure you are not willing to expand your package in which classical parameters will be used instead?

For consistency and stability I only include classical IRT parameterization when they matter (such as in rating scale models). If you must use the classical version of this model, the easiest solution is just to write your own customized item via createItem() and go from there. The documentation actually already has an example of how to do with the classical 2PLM, so that should be enough to get you started. HTH.

Phil

 

To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/ec7a2df3-f673-44d6-a9f2-6d7e57f886a9n%40googlegroups.com.

[Monika Vaheoja]

Thanks Phil, 

I think i have it now (alsmost). 

because i need a vcov of a en b parameters and your suggestion does not transform the vcov, and will only give SE for each item seperaterly i will write my own tranformation, which also means that i can run my code as i specified it earlier and just take a and b of classicalIRT parms, vcov of a and d parameters, and transform the vcov by hand. in which i take along that: 

a = a

d = -b * a

thus b = d/-a

thank you for your answers. 

Monika