Direction of wording issues

[Conal Monaghan]

Hi all,

           I am having some issues with negatively worded items (items worded in the opposite direction to the construct; contrait) influencing the scale much more strongly that positively (protrait) worded ones (Graded response model on a 7-point Likert with 8 items, roughly 1000 participants). This is likely due to a method effect, yet the scree plot (attached as "Rplot") suggests a single factor and EFA / CFA support a 1 factor solution as fitting the data well. LD was satisfied with no Q3 stat exceeding |.30|. Item fit also seems pretty good (see table below). The information from both positive and negative items is roughly equal (based on information curves) until I reduce the item pool from 20 to 10 items - whereupon the negatively worded ones dominate the information space despite the final 10 items having similar curves before reduction. This is particularly evident when looking at the final information curves (see attached Figure "Picture1"). Before resigning to the fact that we cannot explain this from a unidimensional IRT model, is there anything I should pursue regarding this issue or can mirt() handle this without explicitly running a two-dimensional model?

>  Factor.1.IRT <- mirt(Data, SE= TRUE, model = 1, type = "graded", printcycles = FALSE) 

>  round(cbind(coef(Factor.1.IRT, simplify=TRUE)$item ,(itemfit(Factor.1.IRT))[,2:4]), 3)

IRT Parameter Estimates and Item Fit 

	Information and Threshold Parameters										Item Fit
Item	a	1	2	3	4	5	6			S-X2		DF	p
1	2.28	-2.65	.33	1.93	2.95	5.31	7.86			98.55		82	.10
2	2.06	-2.21	.36	1.59	2.58	4.94	7.39			100.15		86	.14
3R	1.66	-1.91	.82	2.20	3.63	5.19	7.38			107.69		87	.07
4R	1.73	-3.14	-.62	.70	1.50	3.00	5.28			135.00		101	.01
5R	1.98	-3.26	-.36	1.10	2.81	4.40	7.60			74.22		87	.83
6	1.43	-2.62	-.38	.67	1.32	3.43	6.39			89.37		103	.83
7	1.56	-3.09	-.10	.07	1.08	2.89	5.43			129.04		99	.02
8R	1.34	-1.99	-.09	.96	1.77	3.13	5.43			124.78		109	.14

Note: R indicates reverse worded items

 Regards and thanks in advance,

       Conal Monaghan

[Seongho Bae]

Dear Conal,

In my opinion, you may check dimensionality using the DIC or AIC fit statistic first with 20 items. If you can extract the DIC (or AIC) in one-factor model and two-factor model, you can check difference which one bigger. If one-factor model's DIC (or AIC) is bigger than two-factor model's DIC (or AIC), it may support two-factor solution is appropriate than the one-factor solution. And in Item response theory tradition, validity is justified by item local independence statistic, as known as 'item fit statistic'; not item information. So, you may consider S-X2 statistic and Zh statistic to reduce your items when you decide n-factor solution in your model.

If you can't understand what I say, you can check these articles.

Drasgow, F., Levine, M. V., & Williams, E. A. (1985). Appropriateness measurement with polychotomous item response models and standardized indices. British Journal of Mathematical and Statistical Psychology, 38(1), 67-86.

Kang, T., & Chen, T. T. (2008). Performance of the Generalized S‐X2 Item Fit Index for Polytomous IRT Models. Journal of Educational Measurement, 45(4), 391-406.

Reise, S. P. (1990). A comparison of item- and person-fit methods of assessing model-data fit in IRT. Applied Psychological Measurement, 14, 127-137.

Preacher, K. J., Zhang, G., Kim, C., & Mels, G. (2013). Choosing the optimal number of factors in exploratory factor analysis: A model selection perspective. Multivariate Behavioral Research, 48(1), 28-56.

Seongho


2016년 7월 19일 화요일 오후 1시 31분 8초 UTC+9, Conal Monaghan 님의 말: