Bifactor model – possible heywood case

[Nicole Sugden]

Hi,

I’m hoping someone might be able to help me with the following 3 factor bifactor model. I’m conducting a review of multiple models of the 14-item Hospital anxiety depression scale, with a sample size of 347 participants.

It looks like it could be a Heywood case as I received the error message about negative lv variances. I’ve attached the output.

As I’m just reviewing the models, my question is, if this is not a good model and can’t be used as it is, what is the best way to describe what has happened with the model here? Alternatively, are there any modifications that could be made to the model to make it work?

Many thanks.

Here is the code:

Norton3 <-'anxiety=~HADS_woundup_1+HADS_happen_3+HADS_mind_5+HADS_stomach_9+HADS_panic_13

                    depression=~HADS_enjoy_2+HADS_things_4+HADS_cheerful_6+HADS_down_8+HADS_appearance_10+HADS_lookfward_12

                    restlessness=~HADS_relaxed_7+HADS_move_11+HADS_prog_14

          g =~ HADS_woundup_1+HADS_happen_3+HADS_mind_5+HADS_relaxed_7+HADS_stomach_9+HADS_move_11+HADS_panic_13 +

                    HADS_enjoy_2+HADS_things_4+HADS_cheerful_6+HADS_down_8+HADS_appearance_10+HADS_lookfward_12+HADS_prog_14'

 

 

fit<-cfa(Norton3,data=HADS_Nicole1,orthogonal=TRUE,ordered=c("HADS_woundup_1", "HADS_enjoy_2", "HADS_happen_3", "HADS_things_4",

                                                             "HADS_mind_5", "HADS_cheerful_6", "HADS_relaxed_7", "HADS_down_8",

                                                             "HADS_stomach_9", "HADS_appearance_10", "HADS_move_11", "HADS_lookfward_12",

                                                             "HADS_panic_13", "HADS_prog_14"))

 

summary(fit,fit.measures=TRUE,standardized=TRUE,rsquare=TRUE,modindices=TRUE)

 

##########################################################################################

[Jošt Bartol]

Dear Nicole,

I checked your results, and while I cannot say with certainty what causes such results, the following thins are intriguing:

1. Item HADS_woundup_1 has a very low loading.

2. Indicators of factor Anxiety are all NON significant.

3. Indicators for factor Restlessness have different signs (one item + other two -) and the two items are insignificant.

4. Looking at the variances, the factors Anxiety and Restlessness have insignificant error variances.

5. Restlessness has negative error variance -- although I think this is the least of your problems.

So all of these would suggest to me that the bifactor solution is not really appropriate as the general factor "g" explains most of variance in the factors Anxiety and Restlessness. Below I attach a brief description of a similar problem we encountered in one of our studies. See also the references.

We tested the fit of the 12-item bifactor model with a general factor and two specific factors, as found with EFA.  Unfortunately, the results did not support such bifactor model on Sample 2. Fitting the bifactor model where all the items loaded on one general factor and two specific factors determined the items as obtained from EFA, resulted in negative variance for the item X. After dropping this item, we obtained a good fitting model. However, the loadings on the second specific factor were small and some of them were also negative (in absolute terms), ranging from -0.473 to 0.287 with three of them being nonsignificant (at p < .05). Moreover, the factor variances of both specific factors were not significant (at p < .05). According to Chen et al. (2006) and Bornovalova et al. (2020) such results indicate that the second specific factor is not coherent and that its content is not separable from the general factor. When we dropped the second specific factor, and obtained an incomplete bifactor model (Brown, 2015), the error variance of the remaining specific factor was again nonsignificant. If we dropped both specific factors and tested a 11-item unidimensional model, the results did not show adequate fit of the model. Brown (2015) has also suggested that fitting a bi-factor model is possible only when a two-factor model has a good fit with data (which was not the case in our study).

 

***

Bornovalova MA, Choate AM, Fatimah H, Petersen KJ, Wiernik BM. Appropriate use of bifactor analysis in psychopathology research: appreciating benefits and limitations. Biol Psychiatry 2020;88(1).

Brown, TA. Confirmatory factor analysis for applied research. New York, NY, USA: Guilford publications; 2015.

Chen FF, Stephen GW, Karen H. A comparison of bifactor and second-order models of quality of life. Multivariate Behav Res 2006;41(2).

V V pet., 25. nov. 2022 ob 06:17 je oseba Nicole Sugden <csubi...@gmail.com> napisala:

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/e6acf2ee-7981-4f41-b72e-0c583b951a47n%40googlegroups.com.

[Nicole Sugden]

Dear Jost,

Thank you for your detailed reply and example, this is exactly the information I needed to understand what might be happening with this model. 

I also have a 2 factor bifactor model (depression and anxiety) which might have a similar issue? Item 1 has a low loading, and none of the anxiety items are significant.

I calculated the ECV for the general factor, this was 68.7. Omega hierarchical was .76.

I've attached the output below also. We also tested a single level 2 factor model that had good fit

Would it be appropriate to conclude that the 3 factor model which had the negative variance/nonsignificant loadings is not a suitable model? Also, that the 2 factor bifactor model had good fit but that anxiety is not really separated from the general factor, so might also not be a great model? Instead, the 2 factor (single level) model might be more suitable?

Any thoughts on this would be great, thanks.

Nicole

[Jošt Bartol]

Dear Nicole,

I suppose you could make such conclusions.

I am not familiar with the theoretical background of your study, but I wonder which model would be most appropriate based on theory? Are you conceptualizing anxiety and depression as distinct constructs or are they highly correlated and there is a higher order factor that accounts for both? If the first I would think that the first-order two-factor model is sufficient, if the latter, then the second-order model would be more appropriate. The bifcator models would be appropriate if you assume that the specific factors (anxiety and depression) do not share any variance except that which is accounted for by the general factor. (You might want to check out Reise et al., 2010, regarding which models are appropriate for which kind of conceptualization). Hence, I would choose the model based on theoretical considerations (if such a model is of course empirically supported).

By the way, I see that you removed the factor restlessness -- did you also consider a first-order model with all three factors (anxiety restlessness, depression)?

In addition, note that there are also 'incomplete' bifactor models (see reference to Chen et al. and also Brown in my previous mail). This might be useful if your conceptualization assumes a bifactor model.

Hope this helps!

Jošt

Ps. You can also check other papers by Reise. Has many on bifactor models.

Reise, S. P., Moore, T. M., & Haviland, M. G. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of personality assessment, 92(6), 544-559.

V V pet., 2. dec. 2022 ob 08:17 je oseba Nicole Sugden <csubi...@gmail.com> napisala:

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/774a470a-ea55-4110-b169-9c542757246bn%40googlegroups.com.

[Nicole Sugden]

Dear Jost,

Thanks again for your fast reply.

We are testing a couple of theoretical alternatives (e.g., is there a higher order structure with correlated factors, or are they separate constructs), so I'll have to do some thinking about how to write up that conclusion.

We did look at a 3 factor (single level) model with restlessness, but it didn't fit well at all, and reliability was extremely poor.

Thanks very much for the reading suggestions, I'll have a look at those.

Thank you again for your help, you have been a huge help!

Thanks, Nicole