SEM starting point mirtCAT

[aide...@gmail.com]

Hi Phil, 

I have a quick question regarding the starting point of the SEM in the mirtCAT().

If I used item parameters that were calibrated with mirt(), the starting theta_SE is not 1. If I convert item parameters calibrated from some other software, and save it as mirt.object, then the SEM starts at 1. Can I change the starting theta_SE manually? 

Kind regards,

Aiden

[Felix Fischer]

hi aiden,

you maybe can change the SE of the mirt.object by changing the mod2values(model) table (set the SE to the intended value and set est to FALSE) and then generate a new object:

pars = mod2values(model)

#do you changes

model2 = mirt(data,1, pars = pars)

best, felix

[aide...@gmail.com]

Hi Felix, 

Thanks for the advice, but I can't see where in the pars table that I can change the SEM to 1. 

In fact, I'm not even sure why the SEM is not starting at 1. If I save the coef() and resave the parameters using generate.mirt_object() then the starting SEM value is 1.  Just trying to figure it out. 

>       result$thetas_SE_history

       Theta_1

[1,] 3.0386927

[2,] 1.7616820

[3,] 1.3058708

[4,] 1.1807706

[5,] 1.1096125

[6,] 1.0625951

[7,] 1.0296049

[8,] 0.8336817

I would appreciate any more advice. 

Kind regards,

Aiden

[Felix Fischer]

Hi,

i remember that i did specify the SE at the start of a CAT once, but I might have been using catR then. As far as i see it seems that mirtCAT doesn't let you specify the SE at the start of the CAT.

regarding the SEM not starting at one - i wouldn't expect that as well - can you provide a minimal example?

best, felix

[aide...@gmail.com]

Hi Felix, 

Thanks for helping out. 

Here is a reproducible code. Though this time the starting theta is less than 1. 

This only happens with the Rasch model though. If I estimate it with other 2pl models, then the SEM beings at 1. 

Probably I don't fully understand how Rasch performs in CAT. Some guidance would be much appreciated! 

Kind regards,

Aiden

#make some data

set.seed(1234)

a <- matrix(rep(1, 10))

d <- matrix(c(1,0.5,-.5,-1), 10, 4, byrow = TRUE)

c <- seq(-1, 1, length.out=10)

data <- simdata(a, d + c, 2000, itemtype = rep('graded',10))

# estimate Rasch model

mod1 <- mirt(data, 1, itemtype = 'Rasch')

# mod1 <- mirt(data, 1, itemtype = 'graded') # sem starts at 1 for graded/grsm/gpcm

# generate pattern

set.seed(1)

Theta = matrix(sample(fscores(mod1, method = "EAP"), size = 1000))

pattern = generate_pattern(mod1, Theta = as.matrix(Theta))

# stopping rule

design = list(min_SEM = 0.2, min_items = 1, max_items = 7)

# CAT

result = mirtCAT(mo = mod1, method="EAP", criteria = "MI", start_item = 'MI', 

                 local_pattern = pattern[3,], design = design, progress = TRUE)

# theta history

result$thetas_history

# sem history

result$thetas_SE_history

[Felix Fischer]

Hi Aiden,

 

when i run your code, i get SE < 1.

 

best, felix

 

> result$thetas_history

         Theta_1

[1,]  0.00000000

[2,]  0.21805821

[3,]  0.21406588

[4,]  0.08230441

[5,]  0.14839043

[6,]  0.17393387

[7,]  0.05247185

[8,] -0.01027411

> 

> # sem history

> result$thetas_SE_history

       Theta_1

[1,] 0.3645817

[2,] 0.3236480

[3,] 0.2915845

[4,] 0.2676246

[5,] 0.2477136

[6,] 0.2320242

[7,] 0.2207434

[8,] 0.2107458

 

> sessionInfo()

R version 4.0.2 (2020-06-22)

Platform: x86_64-w64-mingw32/x64 (64-bit)

Running under: Windows 10 x64 (build 18362)

 

Matrix products: default

 

locale:

[1] LC_COLLATE=German_Germany.1252  LC_CTYPE=German_Germany.1252   

[3] LC_MONETARY=German_Germany.1252 LC_NUMERIC=C                   

[5] LC_TIME=German_Germany.1252    

 

attached base packages:

[1] stats4    stats     graphics  grDevices utils     datasets  methods  

[8] base     

 

other attached packages:

[1] mirtCAT_1.10    shiny_1.6.0     mirt_1.33.2     lattice_0.20-41

 

loaded via a namespace (and not attached):

 [1] Rcpp_1.0.6            magrittr_2.0.1        cluster_2.1.0        

 [4] splines_4.0.2         MASS_7.3-51.6         xtable_1.8-4         

 [7] Deriv_4.1.3           R6_2.5.0              rlang_0.4.10         

[10] fastmap_1.1.0         pbapply_1.4-3         dcurver_0.9.2        

[13] tools_4.0.2           parallel_4.0.2        grid_4.0.2           

[16] lpSolve_5.6.15        nlme_3.1-148          mgcv_1.8-31          

[19] vegan_2.5-7           ellipsis_0.3.1        htmltools_0.5.1.1    

[22] digest_0.6.27         lifecycle_1.0.0       permute_0.9-5        

[25] GPArotation_2014.11-1 Matrix_1.2-18         later_1.1.0.1        

[28] promises_1.2.0.1      mime_0.10             compiler_4.0.2       

[31] httpuv_1.5.5         

[Aiden]

Hi Felix,

Thanks for testing out the code. 

That behaviour is correct because I am using values from the made-up example. I can’t share the actual data which shows that the starting sem is > 1. In either cases, greater or less than 1, shouldn’t it start at 1? Or rather, I am trying to understand why SEM isn’t starting at 1 for the Rasch model and how can I ensure it starts at 1? 

Kind regards,

Aiden

On 30 Mar 2021, at 17:06, Fischer, Felix <Felix....@charite.de> wrote:



---

Von: mirt-p...@googlegroups.com <mirt-p...@googlegroups.com> im Auftrag von aide...@gmail.com <aide...@gmail.com>
Gesendet: Dienstag, 30. März 2021 13:16:44
An: mirt-package
Betreff: [ext] Re: SEM starting point mirtCAT

 

--
You received this message because you are subscribed to a topic in the Google Groups "mirt-package" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mirt-package/ciolUHgnnII/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mirt-package...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/a517e61d-fceb-4a9c-84c0-18eb53dcac6dn%40googlegroups.com.

[Felix Fischer]

Hi,

I'd agree that it should start at 1. anything odd about your specific itemparameters and/or response patterns? can't really say much without being able to replicate the actual problem.

best, felix

[aide...@gmail.com]

Hi Felix, 

The same example code would show that the SEM is not starting at 1, as you have shown above. 

In this case, how would you prepare the code so that it begins from 1? 

In the bold line below, If you changed the itemtype model to other item types (i.e. graded/grsm/gpcm), then the SEM starts at 1. 

So I suspect it is not the case of the response patterns, and wonder if this is a symptom of the Rasch model that doesn't allow the SEM to start at 1 in the simulation? 

I'm not sure. I would appreciate any advice. 

Kind regards,

Aiden

#make some data

set.seed(1234)

a <- matrix(rep(1, 10))

d <- matrix(c(1,0.5,-.5,-1), 10, 4, byrow = TRUE)

c <- seq(-1, 1, length.out=10)

data <- simdata(a, d + c, 2000, itemtype = rep('graded',10))

# estimate Rasch model

mod1 <- mirt(data, 1, itemtype = 'Rasch')

# mod1 <- mirt(data, 1, itemtype = 'graded') # sem starts at 1 for graded/grsm/gpcm

# generate pattern

set.seed(1)

Theta = matrix(sample(fscores(mod1, method = "EAP"), size = 1000))

pattern = generate_pattern(mod1, Theta = as.matrix(Theta))

# stopping rule

design = list(min_SEM = 0.2, min_items = 1, max_items = 7)

# CAT

result = mirtCAT(mo = mod1, method="EAP", criteria = "MI", start_item = 'MI', 

                 local_pattern = pattern[3,], design = design, progress = TRUE)

# theta history

result$thetas_history

# sem history

result$thetas_SE_history

[Felix Fischer]

If my code would show a SE > 1, I'd assume there is an error somewhere. I'd probably try to estimate EAP score and SE by hand for the first item to see whether i'd get the same estimates.

[aide...@gmail.com]

Probably wouldn't prefer to do it by hand since the calculation might not be consistent with fscores() and may create more confusion. 

So I would like to understand why the SEM isn't starting at 1 when the items are estimated under the Rasch model. 

Thanks, 

Aiden

[Phil Chalmers]

I believe this question has been resolved, but just for clarity here's what is going on. When you estimate a Rasch model the parameterization use by mirt is by default is to fix the slopes to 1 in each item and freely estimate the latent variance (this is equivalent to estimating all the slopes but constraining them to be equal, while leaving the latent variable scale at 1). Hence, knowing nothing about the person's response pattern a prior under this parameterization, and assuming the population of latent traits is normal, gives the prior guess of N(0, $\sigma$), where $\sigma$ is the estimated variance from the Rasch model. This is why you see the starting Theta_SE be equal to   0.3645817, because from the output

> coef(mod1, simplify=TRUE)$cov

> 0.133

and taking the square-root of this variance gives sqrt(0.133) = 0.3646917. Basically, this CAT session would have much tighter theta values estimated for everyone, but that's just a function of the initial scaling and doesn't influence the measurement properties. If you want to keep the scaling fixed to 1 for ease of interpretation, and for comparison with the 2PL, then estimate the Rasch model using the slope-equality constraint version and supply that to mirtCAT(). 

Why might this be important? Priors are influenced by parameterization, so if you aren't careful then you may be performing mis-matched distribution scales, depending on your setup (though for MI searches and such that don't use priors this issue is moot). HTH.

Phil

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/77eb6ae2-b12c-47a5-bf5e-6174e987413fn%40googlegroups.com.

[Phil Chalmers]

I believe I answered this in the previous thread, but let me know if you have further questions. Cheers.

Phil

You received this message because you are subscribed to the Google Groups "mirt-package" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/c979e644-901f-451f-b53f-65aeb28c7f72n%40googlegroups.com.

[aide...@gmail.com]

Hi Phil, 

Thanks so much for explaining. Thanks, Felix for your help thus far. 

Looking at the help file for mirt(), is this the right way to estimate the Rasch model using the slope-equality constraint version? 

------------------------------------------------------------------------

model='f =  1-10

CONSTRAIN = (1-10,a1)

'

mod1 <- mirt(data, model, itemtype = 'gpcm')

------------------------------------------------------------------------

I tested this model on the mirtCAT() but am getting an error. 

------------------------------------------------------------------------

# generate pattern

set.seed(1)

Theta = matrix(sample(fscores(mod1, method = "EAP"), size = 1000))

pattern = generate_pattern(mod1, Theta = as.matrix(Theta))

# stopping rule

design = list(min_SEM = 0.2, min_items = 1, max_items = 7)

# CAT

result = mirtCAT(mo = mod1, method="EAP", criteria = "MI", start_item = 'MI', 

                 local_pattern = pattern[2,], design = design, progress = TRUE)

 |                                                  | 0 % ~calculating  Error in tmp[, paste0("F", 1L:test@nfact), drop = FALSE] : 

  subscript out of bounds

------------------------------------------------------------------------

Kind regards,

Aiden

[Phil Chalmers]

Strange, I just tried your code with the following data generation and couldn't reproduce the issue. Could you modify the following until the error is reproducible?

#####

library(mirt)
library(mirtCAT)

a <- matrix(rlnorm(10,.2,.3))

# for the graded model, ensure that there is enough space between the intercepts,
# otherwise closer categories will not be selected often (minimum distance of 0.3 here)
diffs <- t(apply(matrix(runif(10*4, .3, 1), 20), 1, cumsum))
diffs <- -(diffs - rowMeans(diffs))
d <- diffs + rnorm(10)

data <- simdata(a, d, 1000, itemtype = 'graded')

model='f =  1-10
CONSTRAIN = (1-10,a1)
'
mod1 <- mirt(data, model, itemtype = 'gpcm')

# generate pattern
set.seed(1)
Theta = matrix(sample(fscores(mod1, method = "EAP"), size = 1000))
pattern = generate_pattern(mod1, Theta = as.matrix(Theta))

# stopping rule
design = list(min_SEM = 0.2, min_items = 1, max_items = 7)

# CAT
result = mirtCAT(mo = mod1, method="EAP", criteria = "MI", start_item = 'MI',
                 local_pattern = pattern[2,], design = design, progress = TRUE)

#####  

Phil

To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/0dab69df-437d-4803-b989-ff8ec656554en%40googlegroups.com.

[aide...@gmail.com]

Hi Phil, 

Thanks for your response. 

I've tested on 2 mac computer, my colleague's mac computer and a remote Windows computer, and I am still getting the error for some reason.  

This is a screenshot. Please let me know how else I can help. 

Kind regards,

Aiden