Modelling Method Factors

[Conal Monaghan]

Hi all!

       I apologise as this request is somewhat similar to other posts, however I am having trouble applying the answers to my situation. I am looking for the best way to model method factors between positively and negatively worded items. This is a very regular occurrence in scale development. To that end, I wish to model the method factors underlying the positively worded items, and the negatively worded items. To do this, I estimated all error covariances between the positively worded items, and then estimated all error covariances between the negatively worded items. This should estimate the unique variance associated with wording direction. However, when I run this I receive the error:

Warning message:

In lav_model_vcov(lavmodel = lavmodel, lavsamplestats = lavsamplestats,  :

  lavaan WARNING: could not compute standard errors!

  lavaan NOTE: this may be a symptom that the model is not identified.

This is likely because of the large number of covariance estimates. Therefore, what I have seen on other posts is that the best solution is to constrain either one error covariance within each method factor, or to constrain all covariances within each method factor to be equal. The lavaan website offers solutions to constraining estimates. My question would be whether either solution is appropriate and what the best way to go about constraining would be?

Thanks in advance for any help, and I have attached the model specification below and the respective semPlot.

Model <-  ' Factor =~ GENAT_1 + GENAT_2 + GENAT_3 + GENAT_4 + GENAT_5 + GENAT_6 + GENAT_7 + GENAT_8 + 

GENAT_9_R + GENAT_10_R + GENAT_11_R + GENAT_12_R + GENAT_13_R + GENAT_14_R + GENAT_15_R + GENAT_16_R

# Estimate the error covariances between positively worded items

GENAT_1 ~~ GENAT_2

GENAT_1 ~~ GENAT_3

GENAT_1 ~~ GENAT_4

GENAT_1 ~~ GENAT_5

GENAT_1 ~~ GENAT_6

GENAT_1 ~~ GENAT_7

GENAT_1 ~~ GENAT_8

GENAT_2 ~~ GENAT_3

GENAT_2 ~~ GENAT_4

GENAT_2 ~~ GENAT_5

GENAT_2 ~~ GENAT_6

GENAT_2 ~~ GENAT_7

GENAT_2 ~~ GENAT_8

GENAT_3 ~~ GENAT_4

GENAT_3 ~~ GENAT_5

GENAT_3 ~~ GENAT_6

GENAT_3 ~~ GENAT_7

GENAT_3 ~~ GENAT_8

GENAT_4 ~~ GENAT_5

GENAT_4 ~~ GENAT_6

GENAT_4 ~~ GENAT_7

GENAT_4 ~~ GENAT_8

GENAT_5 ~~ GENAT_6

GENAT_5 ~~ GENAT_7

GENAT_5 ~~ GENAT_8

GENAT_6 ~~ GENAT_7

GENAT_6 ~~ GENAT_8

GENAT_7 ~~ GENAT_8

# Estimate the error covariances between negatively worded items

GENAT_9_R ~~ GENAT_10_R

GENAT_9_R ~~ GENAT_11_R

GENAT_9_R ~~ GENAT_12_R

GENAT_9_R ~~ GENAT_13_R

GENAT_9_R ~~ GENAT_14_R

GENAT_9_R ~~ GENAT_15_R

GENAT_9_R ~~ GENAT_16_R

GENAT_10_R ~~ GENAT_11_R

GENAT_10_R ~~ GENAT_12_R

GENAT_10_R ~~ GENAT_13_R

GENAT_10_R ~~ GENAT_14_R

GENAT_10_R ~~ GENAT_15_R

GENAT_10_R ~~ GENAT_16_R

GENAT_11_R ~~ GENAT_12_R

GENAT_11_R ~~ GENAT_13_R

GENAT_11_R ~~ GENAT_14_R

GENAT_11_R ~~ GENAT_15_R

GENAT_11_R ~~ GENAT_16_R

GENAT_12_R ~~ GENAT_13_R

GENAT_12_R ~~ GENAT_14_R

GENAT_12_R ~~ GENAT_15_R

GENAT_12_R ~~ GENAT_16_R

GENAT_13_R ~~ GENAT_14_R

GENAT_13_R ~~ GENAT_15_R

GENAT_13_R ~~ GENAT_16_R

GENAT_14_R ~~ GENAT_15_R

GENAT_14_R ~~ GENAT_16_R

GENAT_15_R ~~ GENAT_16_R

'  

Thanks for any help you can provide

[Terrence Jorgensen]

I would only recommend the error-covariances approach when you only have a few items per method (3 or less), because that is statistically equivalent to actually modeling the method factor(s), in the sense that you estimate as many parameters and get the same model-implied covariance matrix.  In your case, you have several items to measure each method factor, so I would recommend actually modeling the method factors, using a much more parsimonious bifactor model:

Model <-  ' 

Factor =~ GENAT_1 + GENAT_2 + GENAT_3 + GENAT_4 + GENAT_5 + GENAT_6 + GENAT_7 + GENAT_8 + 

          GENAT_9_R + GENAT_10_R + GENAT_11_R + GENAT_12_R + 

          GENAT_13_R + GENAT_14_R + GENAT_15_R + GENAT_16_R

Pos =~ GENAT_1 + GENAT_2 + GENAT_3 + GENAT_4 + GENAT_5 + GENAT_6 + GENAT_7 + GENAT_8

Neg =~ GENAT_9_R + GENAT_10_R + GENAT_11_R + GENAT_12_R + 

       GENAT_13_R + GENAT_14_R + GENAT_15_R + GENAT_16_R

Factor ~~ 0*Pos + 0*Neg

'

If you are unfamiliar with bifactor models, it is pretty easy to Google around for basic information about them and find references.  For instance, you might also need to constrain the Pos~~Neg covariance to zero if there are still identification issues, although I'm pretty sure you can estimate it (and expect it to be negative).  But method factors should be uncorrelated with general/trait factors.  You can also search the SEMNET archive for more discussions about bifactor models and MMTM modes, since this isn't a lavaan-specific issue.

Terry

[Conal Monaghan]

Hi Terry,
Thank you for your help. I modelled one additional method factor with paths to the negative items only. This fitted the data well and an approach I would recommend to others. Additionally, I found several papers recommending the negative method latent factor approach if others reading this are interested (email me).

On a side note, would you happen to know of any research concerning which fit indices are applicable to UlS models? I have been told that RMSEA, CFI, and NNFI are OK. However that SRMR is not. Lavaan produces all fit indices under UlS, although several stats packages do not. Thus I am wondering whether certain fit indices are inappropriate or just that other software not being able to calculate SRMR.

Cheers,

Conal Monaghan

[Terrence Jorgensen]

On a side note, would you happen to know of any research concerning which fit indices are applicable to UlS models?  

Are you talking about unweighted least squares (ULS) estimation?

I have been told that RMSEA, CFI, and NNFI are OK. However that SRMR is not. Lavaan produces all fit indices under UlS, although several stats packages do not. Thus I am wondering whether certain fit indices are inappropriate or just that other software not being able to calculate SRMR.

I haven't read any research about fit indices under ULS estimation compared to the same fit indices under MLE, GLS, or WLS.  But I don't see why SRMR couldn't be computed in any software. -- it is just the average standardized difference between observed and model-implied sample moments.  I'm not really a fan of SRMR anyway because a single huge residual should be addressed, even if the average residual is small; thus, I would recommend looking at all standardized residuals.  

I have read that ULS estimation should be your last resort because the SEs are so inefficient, and I think the test statistic might over-reject, too (don't quote me on that).  Why are you using ULS instead of a more efficient estimator?

Terry