Conducting a CFA on bi-factor models

Joan Chan

unread,

Jul 23, 2020, 12:34:41 AM7/23/20

to lavaan

Hi, I would like to conduct a CFA on a bi-factor model of the TAS-20 but have run into some problems interpreting the output. My code and the model fit was good, however, I noticed that I did not get any factor loadings for the EOT subscale, the variance for the EOT subscale was negative, and the R squared values for items 15 and 16 were negative. This is odd because I tested other models on the TAS-20 using the same set of data but haven't come across any negative variances or R squared values. Is this an issue with the code or the data?

This is the code I used:

TAS20_model6 <- 'DIF =~ TAS20_1 + TAS20_3 + TAS20_6 + TAS20_7 + TAS20_9 + TAS20_13 + TAS20_14

DDF =~ TAS20_2 + TAS20_4_R + TAS20_11 + TAS20_12 + TAS20_17

EOT =~ TAS20_5_R + TAS20_8 + TAS20_10_R + TAS20_15 + TAS20_16 + TAS20_18_R + TAS20_19_R + TAS20_20

method =~ TAS20_4_R + TAS20_5_R + TAS20_10_R + TAS20_18_R + TAS20_19_R

gen_alexi =~ TAS20_1 + TAS20_3 + TAS20_6 + TAS20_7 + TAS20_9 + TAS20_13 + TAS20_14 + #DIF

TAS20_2 + TAS20_4_R + TAS20_11 + TAS20_12 + TAS20_17 + #DDF

TAS20_5_R + TAS20_8 + TAS20_10_R + TAS20_15 + TAS20_16 + TAS20_18_R + TAS20_19_R + TAS20_20 #EOT

DIF ~~ DDF

DIF ~~ EOT

DDF ~~ EOT

gen_alexi =~ 0*DIF + 0*DDF + 0*EOT + 0*method'

TAS20_model6_fit <- cfa(TAS20_model6,

data = final_data_sg,

estimator = "MLM",

orthogonal = FALSE,

std.lv = FALSE)

summary(TAS20_model6_fit,

fit.measures = TRUE,

standardized = TRUE,

rsquare = TRUE)

Please let me know if you have any clue what went wrong. Thank you very much!

Patrick (Malone Quantitative)

unread,

Jul 23, 2020, 7:56:13 AM7/23/20

to lav...@googlegroups.com

Joan,

At a guess your biggest problem is that this

gen_alexi =~ 0*DIF + 0*DDF + 0*EOT + 0*method

should be

gen_alexi ~~ 0*DIF + 0*DDF + 0*EOT + 0*method
method ~~ 0*DIF + 0*DDF + 0*EOT

As written, it's specifying a second-order factor (not a bifactor)
with loadings fixed to zero, and isn't constraining the method factor
to be uncorrelated with the content factors.

Once you fix those things, you may continue to have problems with both
gen_alexi and method in the model, but one step at a time.

Pat

> --
> You received this message because you are subscribed to the Google Groups "lavaan" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/ce97f70d-c31d-4eeb-bfaf-a06c6754565ao%40googlegroups.com.

--
Patrick S. Malone, Ph.D., Malone Quantitative
NEW Service Models: http://malonequantitative.com

He/Him/His

Joan Chan

unread,

Jul 24, 2020, 11:30:53 AM7/24/20

to lavaan

Hi Pat, thank you so much for your help. Just another question, I received this error after conducting CFA on a bi-factor model :

Warning message:

In lav_object_post_check(object) :

lavaan WARNING: covariance matrix of latent variables

is not positive definite;

use lavInspect(fit, "cov.lv") to investigate.

I used lavInspect(fit, "cov.lv") to check my covariances and found that I had a negative covariance. I did not receive this error when I conducted other CFAs on the same dataset. In fact, sometimes I received this error even if I do not have any negative covariances. How big of an issue is this and what can I do about it?

Thank you!

Patrick (Malone Quantitative)

unread,

Jul 24, 2020, 1:21:32 PM7/24/20

to lav...@googlegroups.com

Is this based on the code I sent you? Or a new problem? Can you say more about what you're trying to do in this model?

That said, the most common source of this warning (not error) is a sample size that is small relative to the complexity of the model.

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/de73ed44-8eb8-44ee-af3a-486e1a57bf7ao%40googlegroups.com.

Joan Chan

unread,

Jul 24, 2020, 9:07:01 PM7/24/20

to lavaan

I received this warning for a bi-factor model that I am conducting on the PAQ. This is the code that I used:

PAQ_model6 <-'

N_DIF =~ PAQ_2+ PAQ_8+ PAQ_14+ PAQ_20

P_DIF =~ PAQ_5+ PAQ_11+ PAQ_17+ PAQ_23

N_DDF =~ PAQ_1+ PAQ_7+ PAQ_13+ PAQ_19

P_DDF =~ PAQ_4+ PAQ_10+ PAQ_16+ PAQ_22

G_EOT =~ PAQ_3+ PAQ_6+ PAQ_9+ PAQ_12+ PAQ_15+ PAQ_18+ PAQ_21+ PAQ_24

gen_alex =~ PAQ_1 + PAQ_2 + PAQ_3 + PAQ_4 + PAQ_5 + PAQ_6 + PAQ_7 + PAQ_8 + PAQ_9 + PAQ_10 +

PAQ_11 + PAQ_12 +PAQ_13 + PAQ_14 + PAQ_15 + PAQ_16 + PAQ_17 + PAQ_18 + PAQ_19 + PAQ_20 +

PAQ_21 + PAQ_22 + PAQ_23 + PAQ_24

N_DIF ~~ N_DDF

P_DIF ~~ P_DDF

N_DIF ~~ P_DIF

N_DDF ~~ P_DDF

gen_alexi ~~ 0* N_DIF + 0* P_DIF + 0*N_DDF + 0*P_DDF + 0*G_EOT

G_EOT ~~ 0* N_DIF + 0* P_DIF + 0*N_DDF + 0*P_DDF

N_DIF ~~ 0*P_DDF

P_DIF ~~ 0*N_DDF '

And here is the code that I used to test the fit of the model:

PAQ_model6_fit <- cfa(PAQ_model6,

data = final_data_Sg,

estimator = "MLM",

orthogonal = FALSE,

std.lv = FALSE)

My sample sizes are about 400 to 600 and I am trying to figure out which model (amongst the 6 that I have tested for the PAQ) I should use to conduct measurement invariance on the PAQ.

To unsubscribe from this group and stop receiving emails from it, send an email to lav...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/de73ed44-8eb8-44ee-af3a-486e1a57bf7ao%40googlegroups.com.

Prof. Gavin Brown

unread,

Jul 24, 2020, 11:24:19 PM7/24/20

to lavaan

Negative covariance is a big deal and you need to fix it because the model is NOT admissible.

there are a number of strategies

1. there may be an item or latent variable that has negative error variance causing the covariance matrix not to be positive definite. find and correct it to a small number above zero.

2. you potentially have too many factors. look at the correlation values. if close to 1.00 or >1.00 you know those 2 factors need to be joined together

3. force some of the covariances not to be negative --i have the least experience with doing that.

Patrick (Malone Quantitative)

unread,

Jul 25, 2020, 10:04:44 AM7/25/20

to lav...@googlegroups.com

Something I missed in your previous post:

lavaan WARNING: covariance matrix of latent variables

is not positive definite;

use lavInspect(fit, "cov.lv") to investigate.

I used lavInspect(fit, "cov.lv") to check my covariances and found that I had a negative covariance.

Negative covariance is not what this warning means, and negative covariance is not a problem, just like correlations can be negative. A negative variance estimate is more of a concern. Can you provide the results of the lavInspect? And in addition, lavInspect(PAQ_model6_fit, "cor.lv")

And is there a reason for

G_EOT ~~ 0* N_DIF + 0* P_DIF + 0*N_DDF + 0*P_DDF

N_DIF ~~ 0*P_DDF

P_DIF ~~ 0*N_DDF

? How does the model fit with and without those lines?

To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/99e0c3e5-bab2-4a00-b1a1-5ff21cd42fcfo%40googlegroups.com.

Joan Chan

unread,

Jul 26, 2020, 10:31:04 PM7/26/20

to lavaan

Hi, I specifically specified that

G_EOT ~~ 0* N_DIF + 0* P_DIF + 0*N_DDF + 0*P_DDF

N_DIF ~~ 0*P_DDF

P_DIF ~~ 0*N_DDF

because I was using the method that you taught me to test a bi-factor model for the PAQ. Before you taught me that method, I had used gen_alexi =~ 0* N_DIF + 0* P_DIF + 0*N_DDF + 0*P_DDF + 0*G_EOT. However, after I changed it to gen_alex ~~ 0* N_DIF + 0* P_DIF + 0*N_DDF + 0*P_DDF + 0*G_EOT, it allowed N_DIF to correlate with P_DDF, and P_DIF to correlate with N_DDF, which is not what we want in the model. Hence, I added in N_DIF ~~ 0*P_DDF and

P_DIF ~~ 0*N_DDF, to ensure that the model is specified correctly (see the attached picture of the model we are trying to test).

I have copied and pasted the code that I have used and the output below. Hopefully this provides some clarity. Thank you very much for your help.

PAQ_model6_fit <- cfa(PAQ_model6,

+ data = final_data_sg,

+ estimator = "MLM",

+ orthogonal = FALSE,

+ std.lv = FALSE)

Warning message:

In lav_object_post_check(object) :

lavaan WARNING: covariance matrix of latent variables

is not positive definite;

use lavInspect(fit, "cov.lv") to investigate.

> lavInspect(PAQ_model6_fit, "cov.lv")

N_DIF P_DIF N_DDF P_DDF G_EOT gen_lx

N_DIF 0.670

P_DIF 0.030 0.156

N_DDF 0.543 0.000 0.684

P_DDF 0.000 0.127 0.074 0.138

G_EOT 0.000 0.000 0.000 0.000 0.486

gen_alex 0.000 0.000 0.000 0.000 0.000 1.161

> summary(PAQ_model6_fit,

+ fit.measures = TRUE,

+ standardized = TRUE,

+ rsquare = TRUE)

lavaan 0.6-6 ended normally after 104 iterations

Estimator ML

Optimization method NLMINB

Number of free parameters 76

Number of observations 458

Model Test User Model:

Standard Robust

Test Statistic 667.086 442.937

Degrees of freedom 224 224

P-value (Chi-square) 0.000 0.000

Scaling correction factor 1.506

Satorra-Bentler correction

Model Test Baseline Model:

Test statistic 7216.162 5574.476

Degrees of freedom 276 276

P-value 0.000 0.000

Scaling correction factor 1.295

User Model versus Baseline Model:

Comparative Fit Index (CFI) 0.936 0.959

Tucker-Lewis Index (TLI) 0.921 0.949

Robust Comparative Fit Index (CFI) 0.952

Robust Tucker-Lewis Index (TLI) 0.941

Loglikelihood and Information Criteria:

Loglikelihood user model (H0) -17799.832 -17799.832

Loglikelihood unrestricted model (H1) -17466.289 -17466.289

Akaike (AIC) 35751.665 35751.665

Bayesian (BIC) 36065.307 36065.307

Sample-size adjusted Bayesian (BIC) 35824.106 35824.106

Root Mean Square Error of Approximation:

RMSEA 0.066 0.046

90 Percent confidence interval - lower 0.060 0.041

90 Percent confidence interval - upper 0.071 0.051

P-value RMSEA <= 0.05 0.000 0.887

Robust RMSEA 0.057

90 Percent confidence interval - lower 0.049

90 Percent confidence interval - upper 0.064

Standardized Root Mean Square Residual:

SRMR 0.060 0.060

Parameter Estimates:

Standard errors Robust.sem

Information Expected

Information saturated (h1) model Structured

Latent Variables:

Estimate Std.Err z-value P(>|z|) Std.lv Std.all

N_DIF =~

PAQ_2 1.000 0.819 0.475

PAQ_8 1.232 0.147 8.400 0.000 1.008 0.593

PAQ_14 1.121 0.132 8.466 0.000 0.918 0.555

PAQ_20 1.053 0.161 6.552 0.000 0.862 0.528

P_DIF =~

PAQ_5 1.000 0.395 0.242

PAQ_11 2.508 0.750 3.344 0.001 0.992 0.650

PAQ_17 1.377 0.362 3.800 0.000 0.544 0.363

PAQ_23 1.399 0.347 4.031 0.000 0.553 0.381

N_DDF =~

PAQ_1 1.000 0.827 0.476

PAQ_7 1.241 0.152 8.162 0.000 1.026 0.577

PAQ_13 1.372 0.158 8.694 0.000 1.134 0.625

PAQ_19 1.186 0.147 8.083 0.000 0.981 0.565

P_DDF =~

PAQ_4 1.000 0.372 0.221

PAQ_10 2.956 0.965 3.064 0.002 1.099 0.678

PAQ_16 1.259 0.317 3.971 0.000 0.468 0.292

PAQ_22 1.595 0.419 3.810 0.000 0.593 0.359

G_EOT =~

PAQ_3 1.000 0.697 0.407

PAQ_6 1.149 0.204 5.623 0.000 0.801 0.489

PAQ_9 1.139 0.162 7.046 0.000 0.794 0.518

PAQ_12 1.278 0.206 6.212 0.000 0.891 0.521

PAQ_15 1.235 0.222 5.564 0.000 0.860 0.515

PAQ_18 1.468 0.227 6.476 0.000 1.023 0.637

PAQ_21 1.133 0.189 5.997 0.000 0.790 0.498

PAQ_24 1.210 0.197 6.139 0.000 0.843 0.506

gen_alex =~

PAQ_1 1.000 1.078 0.620

PAQ_2 0.980 0.059 16.541 0.000 1.056 0.612

PAQ_3 0.803 0.088 9.079 0.000 0.865 0.506

PAQ_4 1.134 0.093 12.133 0.000 1.222 0.725

PAQ_5 1.025 0.098 10.456 0.000 1.105 0.676

PAQ_6 0.496 0.092 5.390 0.000 0.535 0.327

PAQ_7 0.754 0.077 9.787 0.000 0.813 0.457

PAQ_8 0.862 0.080 10.718 0.000 0.928 0.546

PAQ_9 0.598 0.083 7.193 0.000 0.645 0.421

PAQ_10 0.744 0.100 7.435 0.000 0.801 0.495

PAQ_11 0.778 0.095 8.203 0.000 0.838 0.550

PAQ_12 0.854 0.095 8.959 0.000 0.921 0.538

PAQ_13 0.991 0.086 11.545 0.000 1.067 0.588

PAQ_14 1.023 0.066 15.464 0.000 1.103 0.666

PAQ_15 0.657 0.090 7.299 0.000 0.708 0.424

PAQ_16 1.201 0.095 12.614 0.000 1.294 0.808

PAQ_17 1.051 0.095 11.064 0.000 1.133 0.756

PAQ_18 0.679 0.096 7.063 0.000 0.731 0.455

PAQ_19 0.990 0.073 13.527 0.000 1.067 0.614

PAQ_20 0.925 0.070 13.130 0.000 0.997 0.610

PAQ_21 0.564 0.091 6.191 0.000 0.607 0.383

PAQ_22 1.087 0.105 10.329 0.000 1.171 0.709

PAQ_23 0.871 0.091 9.601 0.000 0.939 0.647

PAQ_24 0.852 0.100 8.485 0.000 0.919 0.551

Covariances:

Estimate Std.Err z-value P(>|z|) Std.lv Std.all

N_DIF ~~

P_DIF 0.030 0.024 1.228 0.220 0.093 0.093

N_DDF ~~

P_DDF 0.074 0.027 2.701 0.007 0.241 0.241

N_DIF ~~

N_DDF 0.543 0.116 4.671 0.000 0.802 0.802

P_DIF ~~

P_DDF 0.127 0.070 1.811 0.070 0.864 0.864

N_DIF ~~

P_DDF 0.000 0.000 0.000

P_DIF ~~

N_DDF 0.000 0.000 0.000

N_DIF ~~