On the issue of model non-convergence in CFA models

[hongyi sun]

Dear all,

I am currently working on a CFA model that includes an uni factor model, a bi factor model, and a corrlated factor model. There are 25 variables in the model, and neither the uni nor the corrlated factor model has any problems (although the fit is not good, as I would expect). However, when I proceeded to the bi-factor model, the model did not converge, and I tried to add or subtract variables from it, but to no avail. Is there a good way to help me troubleshoot the problem of non-convergence of the model step by step? I'm new to CFA, so I'd like to have some details.

[meng hu]

I am also working on bifactor models, and yes this type of model may have more troubles to fit than other models (higher-order factor or correlated factor models). You need to tell us what error message you get precisely and the model you use along with the constraints (which could be the issue). Typically though, the reason for non-convergence could arise from a model being completely misspecified. I usually check the loadings of the variables on each factors through EFA. Sometimes, changing the estimation method such as optim.method="BFGS" could help but it depends on your error message. However, I highly recommend, before jumping into the bifactor, checking what's causing the misfit in the uni factor or correlated factor model. This is done with modificationIndices(), especially check for loadings and residual correlations with very high MI (don't forget though that MI is highly affected by sample size). When you improve your fit to acceptable levels, try to fit the bifactor again using these additional specifications. I wouldn't try to fit a bifactor if simpler models don't fit well even if they convergence, as it shows they are not correctly specified.

[hongyi sun]

Here is the model fit information of Uni factor model：

Estimator DWLS Optimization method NLMINB Number of model parameters 39 Used Total Number of observations 970 15645 Model Test User Model: Standard Scaled Test Statistic 1047.233 808.642 Degrees of freedom 119 119 P-value (Chi-square) 0.000 0.000 Scaling correction factor 1.356 Shift parameter 36.259 simple second-order correction Model Test Baseline Model: Test statistic 3923.977 2377.484 Degrees of freedom 136 136 P-value 0.000 0.000 Scaling correction factor 1.690 User Model versus Baseline Model: Comparative Fit Index (CFI) 0.755 0.692 Tucker-Lewis Index (TLI) 0.720 0.648 Robust Comparative Fit Index (CFI) NA Robust Tucker-Lewis Index (TLI) NA Root Mean Square Error of Approximation: RMSEA 0.090 0.077 90 Percent confidence interval - lower 0.085 0.072 90 Percent confidence interval - upper 0.095 0.082 P-value H_0: RMSEA <= 0.050 0.000 0.000 P-value H_0: RMSEA >= 0.080 0.999 0.197 Robust RMSEA NA 90 Percent confidence interval - lower NA 90 Percent confidence interval - upper NA P-value H_0: Robust RMSEA <= 0.050 NA P-value H_0: Robust RMSEA >= 0.080 NA Standardized Root Mean Square Residual: SRMR 0.154 0.154 Parameter Estimates: Standard errors Robust.sem Information Expected Information saturated (h1) model Unstructured

> modificationindices(fit,sort. = T, minimum.value = 30.00) mi epc sepc.lv sepc.all sepc.nox 124 364.366 0.329 0.329 0.549 0.549 123 226.015 0.326 0.326 0.880 0.880 148 105.898 -0.423 -0.423 -0.709 -0.709 111 59.855 0.638 0.638 0.282 0.282 170 50.760 0.428 0.428 0.583 0.583 94 46.682 0.489 0.489 0.245 0.245 154 46.588 -0.318 -0.318 -0.292 -0.292 178 41.532 0.682 0.682 0.820 0.820 157 40.750 -0.228 -0.228 -0.238 -0.238 156 37.034 0.312 0.312 0.281 0.281 153 33.405 0.230 0.230 0.213 0.213 97 30.953 0.200 0.200 0.174 0.174

> standardizedSolution(fit) est.std se z pvalue ci.lower ci.upper 1 0.198 0.033 5.950 0.000 0.133 0.263 2 0.198 0.030 6.519 0.000 0.139 0.258 3 0.270 0.030 9.075 0.000 0.211 0.328 4 0.299 0.020 15.078 0.000 0.260 0.338 5 0.709 0.043 16.500 0.000 0.624 0.793 6 -0.368 0.023 -16.250 0.000 -0.412 -0.323 7 0.852 0.049 17.469 0.000 0.757 0.948 8 0.635 0.065 9.720 0.000 0.507 0.763 9 0.484 0.041 11.928 0.000 0.404 0.563 10 0.313 0.074 4.225 0.000 0.168 0.458 11 0.213 0.053 4.025 0.000 0.110 0.317 12 0.313 0.039 7.989 0.000 0.236 0.390 13 0.291 0.051 5.726 0.000 0.191 0.390 14 0.119 0.044 2.695 0.007 0.032 0.205 15 0.237 0.046 5.124 0.000 0.146 0.328 16 0.545 0.037 14.865 0.000 0.473 0.617 17 0.449 0.071 6.305 0.000 0.310 0.589

Here is the correlated model:

Estimator DWLS Optimization method NLMINB Number of model parameters 40 Used Total Number of observations 970 15645 Model Test User Model: Standard Scaled Test Statistic 1047.218 811.050 Degrees of freedom 118 118 P-value (Chi-square) 0.000 0.000 Scaling correction factor 1.351 Shift parameter 35.947 simple second-order correction Model Test Baseline Model: Test statistic 3923.977 2377.484 Degrees of freedom 136 136 P-value 0.000 0.000 Scaling correction factor 1.690 User Model versus Baseline Model: Comparative Fit Index (CFI) 0.755 0.691 Tucker-Lewis Index (TLI) 0.717 0.644 Robust Comparative Fit Index (CFI) NA Robust Tucker-Lewis Index (TLI) NA Root Mean Square Error of Approximation: RMSEA 0.090 0.078 90 Percent confidence interval - lower 0.085 0.073 90 Percent confidence interval - upper 0.095 0.083 P-value H_0: RMSEA <= 0.050 0.000 0.000 P-value H_0: RMSEA >= 0.080 1.000 0.248 Robust RMSEA NA 90 Percent confidence interval - lower NA 90 Percent confidence interval - upper NA P-value H_0: Robust RMSEA <= 0.050 NA P-value H_0: Robust RMSEA >= 0.080 NA Standardized Root Mean Square Residual: SRMR 0.154 0.154 Parameter Estimates: Standard errors Robust.sem Information Expected Information saturated (h1) model Unstructured
> modificationindices(fit,sort. = T, minimum.value = 30.00) mi epc sepc.lv sepc.all sepc.nox 143 364.861 0.329 0.329 0.549 0.549 144 225.975 0.326 0.326 0.881 0.881 157 107.775 -0.431 -0.431 -0.723 -0.723 130 59.909 0.638 0.638 0.282 0.282 190 50.874 0.429 0.429 0.586 0.586 163 47.640 -0.324 -0.324 -0.297 -0.297 114 46.693 0.489 0.489 0.245 0.245 166 42.766 -0.238 -0.238 -0.250 -0.250 198 41.529 0.682 0.682 0.820 0.820 165 37.324 0.315 0.315 0.284 0.284 162 33.934 0.235 0.235 0.217 0.217 116 30.967 0.200 0.200 0.174 0.174 Here is the error message when i run the bi factor model:

In lavaan::lavaan(model = Bi.model lavaan WARNING: the optimizer warns that a solution has NOT been found! > fitMeasures(fit,fit.measures="all", baseline.model=NULL) Error in lav_fit_measures lavaan ERROR: fit measures not available if model did not converge

I don't expect the model fit to get better now, my aim is to have the bi factor models converge.

Sincerely hope for help from all of you.

[mh.namele...@gmail.com]

Some of the MI values stick out like a sore thumb, and I would try respecifying the model based on the highest MI values and see if the fit improves (it should). Since I don't see your model specification (syntax or formula), I'm here assuming you have an interpretable model, that is, a model without too many cross loadings and other similar issues related to poor fit. Because seeing such low value of CFI/TLI and high RMSEA makes me think your model is not correctly specified.

[hongyi sun]

Thank you so much for your answer.

i have remove the Variable with high MI, but the bi factor model is still not converge. Thats very odd. If there is anyway i can try to figure out why bi factor model is not converge all the time?

Looking forward to your reply.

Yours,

Hongyi

[Keith Markus]

Hongyi,

It has been a while since I have worked with bi-factor models regularly but, as I recall, bi-factor models can be finicky even when correctly specifed.  Here are some strategies that I might try in your situation, purely off the top of my head.  Hopefully I am not missing something.

1. Your root mean square residual seems as though it may be large.  Look at your correlation residuals. If the matrix is overwhelming, plot the unique non-diagonal correlations in a stem plot (the lower.tri() function is helpful for this).  Then look up the largest residuals and see what variables are involved.  (This suggestion is not specific to bi-factor models.)

2. Fit a one-factor CFA separately for each factor.  Identify sources of poor fit.  Carry over what you learn to the bi-factor model.

3. Fit a one-factor CFA to all the items with correlated uniquenesses in place of the specific factors.  See if anything jumps out.  Possibly use the loading estimates from this model as start values for the general factor loadings in the bi-factor model.

4. Experiment with different methods of identifying the scale of the latent variables.  Try different fixed parameter values.  For fixed loadings, try choosing different variables as index variables.

5. Try fitting the bi-factor model with the general factor loadings constrained or fixed to see if it will converge.  If not, experiment with different values or constraints.  If so, then gradually release the constraints and/or free the fixed values until you encounter a problem.  Try variations to isolate the problem.

I would not expect there to be a simple universal fix that will avoid problems with all bi-factor models.  Instead, I would anticipate a slow, iterative process of trouble-shooting your specific model.  You can apply the same strategies to the next study using a bi-factor model, but what you need to do to reach a converging model may differ from study to study.

Keith

------------------------
Keith A. Markus
John Jay College of Criminal Justice, CUNY
http://jjcweb.jjay.cuny.edu/kmarkus
Frontiers of Test Validity Theory: Measurement, Causation and Meaning.
http://www.routledge.com/books/details/9781841692203/

[hongyi sun]

Thank you for all of your answer, i really appreicate it!  

Now after i trying to add some new variables into the bi factor model. the model is converge now. But at this time, this model had a Heywood case (one estimated variance was negative for one of the lower-order factor,and suggesting not a valid model). So i did not let this variable loading on tthe lower-order factor and only let it loading on the higher- order factor, the model fit is good, but the result still show the estimated variance of this variable is negative (-0.54), However, I have reviewed some literature which says that if the model is a good fit and if the negative estimated variance is relatively small (less than 1 in absolute value), the model is accepted. I would like to ask for your opinion on this issue and how I should interpret this problem in my model.

Looking forward to your reply soon.

Hongyi

[Keith Markus]

Hongyi,

I am not sure that the absolute value of the variance is the critical thing to examine.  It seems plausible to me that you could make the estimates fall within any arbitrary distance from zero by re-scaling your model to have smaller variances.  If the source you consulted suggests otherwise, perhaps you could share a quotation.

Instead, it seems to me that it would be more useful to consider the confidence interval for the negative variance.  This offers a more direct answer to the question of whether the negative variance could be attributable to sampling error.

I believe that Terrance has posted on this topic a number of times over the years.  You may be able to find more in the archive.