M2 (residmat) vs residuals-method

[janine.f...@gmail.com]

Hi Phil,

sorry to bother you again...
Could you tell me what is the difference between the standardized residuals from M2 (residmat) and the ones from the residuals-method (LD,LDG2 or Q3)?
When I compare them there is a difference, but which residuals to choose regarding check of the fit?

Thank you very much for an answer,
best

Janine

[Phil Chalmers]

M2's residuals are based on the linear relationship (in the form of covariance estimates) between the items and how this covariance pattern is "explained" by the IRT model. As such, items must be ordinal by nature in order for this matrix to be of any interpretation use.

The residuals() function returns statistics that are based on the expected values without assuming that covariance is the term which should be explained, but rather the tables of counts directly; hence, LD and such will work with nomial or otherwise non-ordered IRT models. Q3 is a historical statistic really, based on two-step estimates, and probably isn't as optimal in shorter tests. Overall though, they all have generally the same purpose, though residuals() will return p-value estimates while M2() simply gives the residual covariances values.  

Phil

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[janine.f...@gmail.com]

So in my case (input are answers on a Likert-scale) I could take the M2 residuals - even more when I use the SRMSR to compare model fit. Would you say a cut-off of 0.1 would be appropriate and do you know literature that I could cite?

Phil

To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.

[Phil Chalmers]

The residual matrix is in a scaled metric similar to the standard normal, so as long as a quantile of |0.1| is meaningful to you then, sure, why not? 

Phil

To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package+unsubscribe@googlegroups.com.

[janine.f...@gmail.com]

Thank you very much for the quick answer. Maybe to put in other words: when you check for model fit - is there like a "personal" cut-off you use or do you refer on a special article?

[Phil Chalmers]

On Wed, Jan 31, 2018 at 5:46 PM, <janine.f...@gmail.com> wrote:

Thank you very much for the quick answer. Maybe to put in other words: when you check for model fit - is there like a "personal" cut-off you use or do you refer on a special article?

....personal? No. Though I imagine people have, and I'm sure applied researchers have cited them thousands of times. But, I'm not a huge fan of cut-offs (statistical cutoffs, not the pants....though less fond of those too). 

 

To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package+unsubscribe@googlegroups.com.

[Dirk Pelt]

Hi Phil and Janine,

related to this, how exactly would you go about interpreting the residual covariance matrix obtained by residmat = TRUE? Of course we can compare the relative values to see where the model is lacking.

However in absolute terms, given that we are talking about covariances, the values will be dependent on the scale of the items so a cut-off of what is "large" would be impossible right?
Is there a way to standardize them in a way? My goal is to find out what exactly is a "large" residual covariance value.

Do you have any recommendations on rules of thumb or practical guidelines? (Although based on your previous posts you do not seem to like general cut-offs :D)

Dirk

[Phil Chalmers]

On Sat, Mar 3, 2018 at 4:15 PM, Dirk Pelt <dirkp...@gmail.com> wrote:

Hi Phil and Janine,

related to this, how exactly would you go about interpreting the residual covariance matrix obtained by residmat = TRUE? Of course we can compare the relative values to see where the model is lacking.

However in absolute terms, given that we are talking about covariances, the values will be dependent on the scale of the items so a cut-off of what is "large" would be impossible right?
Is there a way to standardize them in a way? My goal is to find out what exactly is a "large" residual covariance value.

The terms in residmat are already standardized, so they can be interpreted as residual correlations estimates (see Maydeu-Olivares, A. & Joe, H. Assessing Approximate Fit in Categorical Data Analysis Multivariate Behavioral Research, 2014, 49, 305-328 for details).

 

Do you have any recommendations on rules of thumb or practical guidelines? (Although based on your previous posts you do not seem to like general cut-offs :D)

I would defer to experts in this field rather than offering cut-offs myself. Perhaps Albert's work gives some insight into suitable cut-offs, though I'm skeptical. 

Phil

 

Dirk