multigroup anchor item specification

[Monika Vaheoja]

Dear Phil, 

Im fitting 2plm irt model to scale two exam forms from different populations in which i need the item parameters to be estimated the same across groups. 

To be sure that i understood your syntax and explanations correcly, i would like to make it sure. that i specified he model correcly: 

Both exams have 25 items (correct/wrong responses) and 5 items (4, 10, 14, 18 and 23). and the respons data has thereby 45 unique items.

  spec = ('F = 1-45, 

            CONSTRAINB = (x4, x10, x14, x18, x23 , a1)')

  

  spec_inv = c('free_means','free_var','intercepts',"slopes")

  

  mirt.tmp <- multipleGroup(rf, model = spec, group = as.factor(df$bkl), 

                                             invariance = spec_inv, SE = TRUE, 

                                             warn=FALSE, message=FALSE, info_if_converged=FALSE, 

                                             SE.type="Oakes") 

But, now i have trouble understanding the aspect how the missing-by-design is handled in multigroup function to estimate variance-covariance matrix? 

in your article (chalmes, 2012, mirt), even though it is out dated (you specify the data in indicator form as 1 and 0). But what happens with the 'missing-by design' in the variance-covariance matrix in multigroup irt? 

i got this question because i have different results when i use SE.type="oaks' and SE.type="crossprod" and the main difference (if i understood it correcly) is the aspect that in Oaks is the fisher information computed as a difference between complete information and missing data information.  

i hope you understand my question about the handling of missing-by-design in multigroup situation when items are contstrained across groups. 

regards, 

Monika Vaheoja

[Phil Chalmers]

Hi Monika,

When fitting a multiple group model you can really just conceptualize these as multi single group models, but where the parameters are allowed to vary conditional on the group membership. If there were to to be p parameters estimated in a single group, then for a two group setup there will be 2*p parameters instead, where the second set is of the same structural type as in the first group (that's not a strict requirement, but it's the most common for reason of DIF testing and such). Hence, the vcov matrix just doubles in size, where the sampling information for each parameter can be based on one group (if unique to a group) or pooled across multiple groups (as is the case with anchor items). The missing by design effect (coded as structural NA values) doesn't really make a difference here, since missing values do not contribute extra information across groups when the missingness is assumed to be MAR. Even if items were constrained across groups, where on group had entirely NAs for a specific item, the NAs do not contribute to the estimation or VCOV, and therefore the constraints across groups becomes entirely based on a single group (i.e., all strength/information is borrowed from one group). 

Regarding your question about the information matrices, the Oakes method is an estimate of the VCOV based on the observed information (negative inverse of the Hessian of the log-likelihood), while the cross-product is also an estimate of the observed information but using the empirical cross product method w.r.t. to the response patterns observed (hence, is not an expected/Fisher estimate). HTH.

Phil

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/50b4869a-9d95-4840-806f-bd49c87fdb08n%40googlegroups.com.

[Monika Vaheoja]

Hi Phil,

Thank you again for your answer. Have you also considered to estimate the item parameters across groups in which you would only estimate parameters per item? because im afraid in the way the parameters are estimated at this point, in multigroup estimation you might be introducing some bias. 

A nice article about this aspect is "Eggen, T. J. H. M., & Verhelst, N. D. (2011). Item calibration in incomplete testing designs. Psicologica, 32(1), 107-132". 

And can you explain exactly, how are the parameter values computed after estimating them seperaterly for each group? What method do you use when the parameter slopes and intercepts are fixed? I cant find any exact literature about it, unfortunatelly.  

And my third question is to clarify what you meant with: " the Oakes method is an estimate of the VCOV based on the observed information (negative inverse of the Hessian of the log-likelihood)". 

I thought that Oaks method is an estimate of the observed information matrix? I'm interested in the observed information matrix from Mirt from multigroup function, do you have any suggestions how could i obtain that matrix directly? 

greetings,

Monika

[Phil Chalmers]

On Wed, Feb 17, 2021 at 11:01 AM Monika Vaheoja <monika...@gmail.com> wrote:

Hi Phil,

Thank you again for your answer. Have you also considered to estimate the item parameters across groups in which you would only estimate parameters per item? because im afraid in the way the parameters are estimated at this point, in multigroup estimation you might be introducing some bias. 

A nice article about this aspect is "Eggen, T. J. H. M., & Verhelst, N. D. (2011). Item calibration in incomplete testing designs. Psicologica, 32(1), 107-132". 

I'm not familiar with this paper, so I'll have to read it during some spare time. Thanks for the reference.

 

And can you explain exactly, how are the parameter values computed after estimating them seperaterly for each group? What method do you use when the parameter slopes and intercepts are fixed? I cant find any exact literature about it, unfortunatelly.  

I don't follow, multipleGroup() doesn't estimate the parameters separately, it does so concurrently based on the complete log-likelihood function summed across the groups. That allows for equality constraints and such, as well as likelihood/Baysian inferences.

 

And my third question is to clarify what you meant with: " the Oakes method is an estimate of the VCOV based on the observed information (negative inverse of the Hessian of the log-likelihood)". 

I thought that Oaks method is an estimate of the observed information matrix? I'm interested in the observed information matrix from Mirt from multigroup function, do you have any suggestions how could i obtain that matrix directly? 

The Oakes method is for the observed information. If you want the direct information component just invert the ACOV estimate using 

Oinfo <- solve(vcov(yourMGmodel))). 

 

To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/86efe60d-4fd5-488e-8d0c-be6f9e4883abn%40googlegroups.com.

[Monika Vaheoja]

Thank you Phil again for your answer. 

Just one last question regarding my question:

And can you explain exactly, how are the parameter values computed after estimating them seperaterly for each group? What method do you use when the parameter slopes and intercepts are fixed? I cant find any exact literature about it, unfortunatelly.  

you said:

I don't follow, multipleGroup() doesn't estimate the parameters separately, it does so concurrently based on the complete log-likelihood function summed across the groups. That allows for equality constraints and such, as well as likelihood/Baysian inferences.

I have this question because, 

when i estimate multigroup 2plm for two groups, both having the same 10 items in exam (which means that there should only be 22 parameters for items and mean and sigma for groups)  then in the output of the mirt following is printed:

Iteration: 66, Log-Lik: -10914.771, Max-Change: 0.00000

Calculating information matrix...

Call:

multipleGroup(data = rf, model = spec, group = as.factor(df$bkl), 

    invariance = spec_inv, SE = TRUE, SE.type = "Oakes", TOL = 1e-07)

Full-information item factor analysis with 1 factor(s).

Converged within 1e-07 tolerance after 66 EM iterations.

mirt version: 1.33.2 

M-step optimizer: nlminb 

EM acceleration: Ramsay 

Number of rectangular quadrature: 61

Latent density type: Gaussian 

Information matrix estimated with method: Oakes

Second-order test: model is a possible local maximum

Condition number of information matrix =  138.8471

Log-likelihood = -10914.77

Estimated parameters: 42 

AIC = 21873.54; AICc = 21874.05

BIC = 21996.76; SABIC = 21926.87

G2 (2024) = 1214.47, p = 1

RMSEA = 0, CFI = NaN, TLI = NaN 

this states that there are 42 parameters estimated. This is the reason why i keep wondering about the way the parameters are estimated for anchor items. 

greetings, 

Monika

[Phil Chalmers]

Hi Monika,

It's hard to determine this issue without some reproducible code. As far as I can tell you should have 22 estimated parameters in a two-group, parameter constrained setup with 10 dichotomous items fit with a 2PLM. When you use something like coef() on your model can you confirm that the parameters were indeed constrained to be equal across the groups? I suspect not, so there may have been typos somewhere in the code, or perhaps a bug is present given the mixture of the invariance and CONSTRAINB specifications (though this is less likely).

Phil

To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/c66b9438-2544-4dae-bb90-7d8c6cc1db21n%40googlegroups.com.