Testing the significance of the difference between 2 theta estimations

[ber...@terluin.net]

To see whether a scale is essentially unidimensional I have created 2 IRT models, a standard unidimensional model and a (better fitting) bifactor model in which local item dependence was captured in specific factors. Subsequently I have saved theta estimations (and their SEs) from the unidimensional model and from the general factor of the bifactor model. Now, I would like to test the intra-individual differences in these theta estimations. 

Suppose one of the subjects had the following thetas and SEs: th1 = 1.0 (se1 = 0.3), th2 = 1.5 (se2 = 0.4)

My idea was to compare the difference between th1 and th2 with the se of this difference, as follows.

thd = th2 - th1 = 1.5 - 1.0 = 0.5

sed = sqrt(se1^2 + se2^2) = sqrt(0.09 + 0.16) = sqrt(0.25) = 0.5

z = thd/sed = 0.5/0.5 = 1

Conclusion: th1 and th2 are not statistically significantly different in this subject.

My question is: is this correct? I realize that my calculation of sed assumes that th1 and th2 are uncorrelated (i.e. independent) estimates. Is that so?

Your answer will be much appreciated.

Berend Terluin

[Phil Chalmers]

Seems interesting, but you also are assuming a good amount of other things in this approach. Namely, that the model parameter precision (i.e., the sampling variability of the item parameters) is not a factor because they are treated as fixed and known, that the theta variables actually do represent the same latent variable (which they might not...as the bifactor is a conditional model), that the SE terms you compute are reasonable representations of the actual sampling variability (which they often aren't if the number of items is too small), and so on. 

Basically, the approach you are taking *might* work, but it rests on a number of potentially unrealistic assumptions.  

Your answer will be much appreciated.

Berend Terluin

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Berend Terluin]

Hi Phil,

 

Thanks a lot. I understand that it is not as simple as I hoped.

Might it be helpful to work with plausible values?

 

Berend Terluin

--

To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

 

Virusvrij. www.avast.com