Nominal model and parameter constrains

[gin...@gmail.com]

Hi! 

I'm trying to model a 12-item unidimensional scale with the following response format: 

1: very easy

2: quite easy

3: quite difficult

4: very difficult

5: I do not know

the opt-out category leads me to apply the nominal model here. However, I think it makes sense to constrain the parameters corresponding to options 1 to 4 to follow an ordinal scale (e.g. constrains acomodating to a cumulative probabilities model), and leaving the parameters corresponding to category 5 to be freely estimated. I have several samples and I'm considering a multigroup setting and a study of DIF, and so checking whether opt-out response functions equialently across groups would be very interesting. 

I have a second issue regarding model GOF. I've fitted the nominal model unconstrained and tried to examine GOF indices via M2(na.rm=TRUE). I get the following error message for all models (same model fitted on different samples): "Error: M2() statistic cannot be calculated due to too few degrees of freedom". My model has 96 estimated parameters and, for example, sample size after row-wise response data removal n=1927 and model@Fit$df = 244140528, so I'm a bit lost here. 

My questions are: 

- can you guide me regarding ordering constrains of parameters for options 1:4?

- any insights on why C2 statistics are not available?

thank you very much in advance,

Regina

[Phil Chalmers]

Hi Regina,

I think setting up the first 1-4 categories with scoring coefficients that mimic the GPCM would make sense (0, 1,2,3, respectively), while leaving the last category as something that is estimable would be the most reasonable. Something like this:

################

> library(mirt)
>
> head(Science)
  Comfort Work Future Benefit
1       4    4      3       2
2       3    3      3       3
3       3    2      2       3
4       3    2      2       3
5       3    4      4       1
6       4    4      3       3
>

> # Item 1 is structured item in question
> baseline_mod <- mirt(Science, itemtype = c('nominal', rep('graded', 3)))
Iteration: 59, Log-Lik: -1607.205, Max-Change: 0.00009
> coef(baseline_mod)[[1]]
          a1 ak0     ak1    ak2 ak3 d0       d1       d2       d3
par 1.071035   0 1.60566 2.0477   3  0 3.912586 6.183833 4.806511
>

> # assume category 1-3 in item 1 are ordered, while the last could be unordered  
> mod <- mirt(Science, itemtype = c('nominal', rep('graded', 3)),
+             model = "Theta = 1-4
+             FIXED = (1, ak0), (1, ak1), (1, ak2),
+             START = (1, ak0, 0), (1, ak1, 1), (1, ak2, 2)
+             FREE = (1,ak3)")
Iteration: 58, Log-Lik: -1608.227, Max-Change: 0.00010
> coef(mod)[[1]]
           a1 ak0 ak1 ak2      ak3 d0       d1      d2       d3
par 0.7146897   0   1   2 3.422804  0 2.583246 4.99544 3.610593
>
> # models effectively the same (more constrained preferred)
> anova(mod, baseline_mod)
                  AIC    SABIC       HQ      BIC    logLik    X2 df     p
mod          3250.453 3264.024 3277.210 3317.965 -1608.227              
baseline_mod 3250.411 3264.780 3278.741 3321.894 -1607.205 2.042  1 0.153

############

Regarding the fit statistics, the df have more to do with the number of items and parameters, where too few items can cause problems as the number of moments is too low. Also note that the na.rm argument is no longer required as the fit statistics now use MAR assumptions (and therefore no row-wise deletion is required). HTH.

Phil

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/mirt-package/4b820439-558e-418d-a44a-4ab4265dd91fn%40googlegroups.com.

[gin...@gmail.com]

Thanks Phil very much for this helpful answer!

I'm using the nominal model for the first time and I'm surprised by how large values my intercept (and naturally also c) parameters show. I reckon the meaning of intercept and slope parameters is not the same as in e.g. the graded response model, but still I wonder do so large values indicate there's something wrong with the model. Item trace lines look about normal, and overall GOF indicators are acceptable and no convergence issues. See below the estimated coefficients: 

IRT_nominal$ms %>% coef(simplify=TRUE, IRTpars=FALSE) %$% items
                       a1 ak0 ak1 ak2 ak3  ak4   d0      d1     d2     d3     d4
it_1  1.85   0   1   2   3 2.14  0  0.2016 -2.199  -5.54 -3.513
it_2  2.40   0   1   2   3 2.26  0  0.1590 -2.556  -6.76 -4.311
it_3  2.21   0   1   2   3 2.05  0  1.0599 -0.656  -4.29 -1.685
it_4  2.70   0   1   2   3 2.29  0 -1.1708 -5.846 -10.50 -7.490
it_5  2.03   0   1   2   3 1.66  0  0.4668 -1.535  -4.51 -0.732
it_6  2.58   0   1   2   3 1.92  0  0.0324 -2.801  -6.71 -2.932
it_7  2.66   0   1   2   3 1.70  0 -0.9838 -5.095  -9.68 -3.946
it_8  2.44   0   1   2   3 1.89  0  0.3228 -2.660  -7.14 -2.729
it_9  3.39   0   1   2   3 2.26  0 -1.0303 -5.762 -12.23 -7.548
it_10 2.55   0   1   2   3 1.95  0 -0.7911 -4.969 -10.06 -5.195
it_11 3.20   0   1   2   3 1.98  0 -0.6084 -4.872 -10.78 -5.450
it_12 2.28   0   1   2   3 2.54  0 -0.3685 -3.398  -7.74 -5.993

The d parameters in the nominal model receive little attention in the literature, in most chapters I've read they're just mentioned as intercepts and warned not to be interpreted directly. Do you think they're in any way useful to the purpose of understanding how response categories work in my model (four categories constrained to mimic Samejima, fith freely estimated)? If so, I'd appreciate some rationale or reference to look into them!

thanks beforehand,

Regina

[Phil Chalmers]

Under this parameterisation the intercepts are more reflective of category endorsement frequencies/probabilities, so the larger negative values have more to do with the observations that few individuals selected those categories. I wouldn't consider this a problem at all, and I imagine if you used something like mirt::itemstats(dat) you'll see very few responses in those large negative intercept categories. 

If it helps, you can play around with the nominal model parameters interactively via mirt::itemplot(shiny=TRUE), which might help you see the meaning of the intercepts when modified. Cheers.

Phil

P.S. If you're interested in reporting standardized information from the NRM I'd recommend installing the dev version as I recently patched this behaviour to be more consistent.

To view this discussion visit https://groups.google.com/d/msgid/mirt-package/ffc0f08a-4f1b-48fb-8ba2-458e66f21838n%40googlegroups.com.

[gin...@gmail.com]

great Phil, thanks again for helping!

Regina

[gin...@gmail.com]

Hi Phil, 

I continued my way through the path of response category order estimation in the context of Bock's model, and found this interesting piece of work with a very simple yet relevant suggestion to improve usefulness of the model for test construction: https://doi.org/10.1177/0013164417714296 

kind regards,

Regina

[Phil Chalmers]

Hi Regina,

Thanks for sharing! I've not come across this paper before, and if I had to approach the issue myself I'd probably make a transformation matrix (C) that would force an ordinal structure from the unconstrained parameter space (I've done this previously in multidimensional non-compensatory modeling contexts). Hopefully the authors did the same.

Phil

To view this discussion visit https://groups.google.com/d/msgid/mirt-package/c1fa317a-66e0-4f70-9d6f-657f39eee58cn%40googlegroups.com.