exploratory multidimensional grm results reporting

[mahmoud hamza]

Hi Phil

Thanks for an awesome package and for your highly appreciated support in this forum.. 

I am developing and validating an assessment tool with 43 questions each with a three level ordinal response (1 - 2 - 3)  using exploratory multidimensional grm. I have 950 respondents and no missing data. 

Kindly I have few scattered questions. 

1- What I understood from your replies to other posts : I should never go for difficulty parameter in multidimensional model as it is can't be understood by human. In addition, raw coefficients from coef() are rarely useful. Instead I should report standardized factor loadings from summary() .. Have I got it right ? 

2- Regarding playing with itemplot using shiny. I find it a great idea. However, I really can't understand the 3D plots of my multidimensional model. Would you please recommend a source to guide me interpreting such plots?

3- Regarding rotation, I understood that you recommend it to be done for factor loadings not coefficients. My multidimensional model has two factors. Their correlation is 0.22. 
I am puzzled whether to use varimax or oblimin. In theory, my factors should be related. 

4- I have chosen unconditional maximum likelihood with EM estimator instead of FIML. Is this right ?

Thank you so much in advance for your help :)

[Phil Chalmers]

On Mon, Jan 27, 2020 at 5:14 AM mahmoud hamza <mah.mo...@gmail.com> wrote:

Hi Phil

Thanks for an awesome package and for your highly appreciated support in this forum.. 

I am developing and validating an assessment tool with 43 questions each with a three level ordinal response (1 - 2 - 3)  using exploratory multidimensional grm. I have 950 respondents and no missing data. 

Kindly I have few scattered questions. 

1- What I understood from your replies to other posts : I should never go for difficulty parameter in multidimensional model as it is can't be understood by human. In addition, raw coefficients from coef() are rarely useful. Instead I should report standardized factor loadings from summary() .. Have I got it right ? 

Multidimensional difficulties are understandable, its just in that they are vectors rather than scalars. A single intercept, much like what you'd find in regression models, often is easier to work with. Raw coefficients ARE useful and meaningful, and the standardization of the slopes (via summary()) can be also be helpful for interpretation in the context of factor analyses.

 

2- Regarding playing with itemplot using shiny. I find it a great idea. However, I really can't understand the 3D plots of my multidimensional model. Would you please recommend a source to guide me interpreting such plots?

Wes Bonifay recently came out with a MIRT book, which is accessible and uses mirt almost exclusively. Christopher Desjardins and Okan Bulut also came out with a book for IRT as well in R. 

 

3- Regarding rotation, I understood that you recommend it to be done for factor loadings not coefficients. My multidimensional model has two factors. Their correlation is 0.22. 
I am puzzled whether to use varimax or oblimin. In theory, my factors should be related. 

The rule of thumb in factor analysis is to always choose oblique rotations. Orthogonal ones are rarely recommended and are available in software these days mostly for historical reasons.

 

4- I have chosen unconditional maximum likelihood with EM estimator instead of FIML. Is this right ?

Sure, why not?

Phil

 

Thank you so much in advance for your help :)

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/8e4a100f-0738-4b53-9e07-727ef0bd50d3%40googlegroups.com.

[mahmoud hamza]

Things are much clearer now. Few more questions please...

1- is there a way to perform an automatic step-wise elimination of items according to item fit in exploratory multidimensional model ? something like stepwiseIt from eRm package but for multidimensional models? 

2- shall I fit a confirmatory MIRT model after the exploratory MIRT. In this case I would use factor loadings to define which items belong to which domain. Regarding the cross loadings, I was considering to add the items to both domains in the confirmatory model, is this ok ? 

3- shall I use drop.zero so as to have 2D dimensional plots, based on the confirmatory model ? 

Sorry if some questions seem trivial, but I am newbie in this field. 

Thank you so much in advance... 

 

On Wednesday, January 29, 2020 at 4:13:54 PM UTC+2, Phil Chalmers wrote:

On Mon, Jan 27, 2020 at 5:14 AM mahmoud hamza <mah.m...@gmail.com> wrote:

Hi Phil

Thanks for an awesome package and for your highly appreciated support in this forum.. 

I am developing and validating an assessment tool with 43 questions each with a three level ordinal response (1 - 2 - 3)  using exploratory multidimensional grm. I have 950 respondents and no missing data. 

Kindly I have few scattered questions. 

1- What I understood from your replies to other posts : I should never go for difficulty parameter in multidimensional model as it is can't be understood by human. In addition, raw coefficients from coef() are rarely useful. Instead I should report standardized factor loadings from summary() .. Have I got it right ? 

Multidimensional difficulties are understandable, its just in that they are vectors rather than scalars. A single intercept, much like what you'd find in regression models, often is easier to work with. Raw coefficients ARE useful and meaningful, and the standardization of the slopes (via summary()) can be also be helpful for interpretation in the context of factor analyses.

 

Ok I understand, I was asking for whether to report intercept and slope for exploratory multidimensional model. If yes, then should I report each parameter only once in its domain ? 

 

2- Regarding playing with itemplot using shiny. I find it a great idea. However, I really can't understand the 3D plots of my multidimensional model. Would you please recommend a source to guide me interpreting such plots?

Wes Bonifay recently came out with a MIRT book, which is accessible and uses mirt almost exclusively. Christopher Desjardins and Okan Bulut also came out with a book for IRT as well in R. 

 

Thanks a lot for the suggestions 

 

3- Regarding rotation, I understood that you recommend it to be done for factor loadings not coefficients. My multidimensional model has two factors. Their correlation is 0.22. 
I am puzzled whether to use varimax or oblimin. In theory, my factors should be related. 

The rule of thumb in factor analysis is to always choose oblique rotations. Orthogonal ones are rarely recommended and are available in software these days mostly for historical reasons.

 

4- I have chosen unconditional maximum likelihood with EM estimator instead of FIML. Is this right ?

Sure, why not?

Phil

 

Thank you so much in advance for your help :)

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mirt-p...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/8e4a100f-0738-4b53-9e07-727ef0bd50d3%40googlegroups.com.

On Wednesday, January 29, 2020 at 4:13:54 PM UTC+2, Phil Chalmers wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to mirt-p...@googlegroups.com.

[Phil Chalmers]

On Wed, Feb 5, 2020 at 7:22 AM mahmoud hamza <mah.mo...@gmail.com> wrote:

Things are much clearer now. Few more questions please...

1- is there a way to perform an automatic step-wise elimination of items according to item fit in exploratory multidimensional model ? something like stepwiseIt from eRm package but for multidimensional models? 

No. And that sounds bizarre to me, and prone to issues. 

 

2- shall I fit a confirmatory MIRT model after the exploratory MIRT. In this case I would use factor loadings to define which items belong to which domain. Regarding the cross loadings, I was considering to add the items to both domains in the confirmatory model, is this ok ? 

You can always fit a confirmatory model after doing exploratory analyses, but like all exploratory frameworks you should try and cross-validate rather than using the same data. Yes, cross-loadings are fine, they just add additional model complexity.

 

3- shall I use drop.zero so as to have 2D dimensional plots, based on the confirmatory model ? 

Up to you. This is only available to help visualize response expected surfaces, and doesn't really add to model fitting information.

Phil

 

To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/1b60be4b-83c8-44b7-9978-6aae53ab8d4a%40googlegroups.com.

[mahmoud hamza]

The book you recommended "MIRT by Wes Bonifay" is really good. Thanks a lot. 

However, I have a single question.. I tried to produce contour plots for individual categories as mentioned in the book for my 2 dimensional grm. The problem is that for 1 out of the 43 questions that I have, the a2 value is 0. I have converted it into 0.01 so as to avoid Inf

In addition, I have changed Wes code accordingly to fit my data by modifying his functions and replacing the arbitrary parameters.  In addition, I have converted intercepts into difficulties

Kindly find attached my trial. 

If anything is not accurate, kindly let me know.. 

In addition, may I ask whether we can include a function in the package for such plot ?

Thank you so much in advance.