model specification second-order

254 views
Skip to first unread message

Franziska Motka

unread,
Apr 8, 2024, 7:49:33 AM4/8/24
to lavaan
Hi everyone, 
this is my first time trying to specify a CFA. My research question is, whether there is a shared underlying mechanism (=sensitization) behind 10 different measures. So I have 10 measures (= manifest variables), divided in two cognitive-behavioral, three psychophysiological and five neural measures. This is what I specified so far in lavaan: 

CFAmodel <- 'cognitivebehavioral =~ AAT_bias_smoking + IAT_app_av_score
             EMG =~ EMGcorr_score + EMGzygo_score + EMGstartle_score
             fMRI =~ ACC + lAG + rThal + rStriatum + lSTG
             sensitization =~ cognitivebehavioral + EMG + fMRI'

Unfortunately, I receive the following error message:

> CFAmodel_a <- cfa(CFAmodel, data_CFA, std.lv = TRUE) Warnmeldung: In lav_model_vcov(lavmodel = lavmodel, lavsamplestats = lavsamplestats, : lavaan WARNING: Could not compute standard errors! The information matrix could not be inverted. This may be a symptom that the model is not identified.

Can somebody help? If I am not wrong, the model should be identified. I was thinking about fixing all paths of the first-order latent variables to 1, because I am only interested in the second-order variable.

Best, 
Franziska

Franziska Motka

unread,
Apr 8, 2024, 9:39:52 AM4/8/24
to lavaan
Update: After more research, I would specify my model as follows: 
CFAmodel <- 'cognitivebehavioral =~ 1*AAT_bias_smoking + IAT_app_av_score
             EMG =~ 1*EMGcorr_score + EMGzygo_score + EMGstartle_score
             fMRI =~ 1*ACC + lAG + rThal + rStriatum + lSTG
             sensitization =~ 1*cognitivebehavioral + EMG + fMRI'

CFAmodel_a <- cfa(CFAmodel, data_CFA, std.lv = TRUE)
summary(CFAmodel_a, fit.measures = TRUE, standardized=TRUE)

Here I receive the warning message:
Warning message: In lav_object_post_check(object) : lavaan WARNING: some estimated ov variances are negative

The variances may have negative estimates due to the low split-half reliability of some measures. If anyone with expertise can provide feedback on whether this model specification makes sense, I would greatly appreciate it. 

Ingo Man

unread,
Apr 9, 2024, 2:57:14 AM4/9/24
to lavaan
Dear Franziska,

if I were you, I would try to reduce the model to localize the error. 
First, are the variable names written correctly?
Second, try to eliminate the std.lv = TRUE statement because it seems you use the marker indicator approach to identifiy the factors (aka give factors the metric of the first variable each). If you use the "fix factor variance to 1" approach, try to eliminate the 1* - statement and leave std.lv = TRUE in the specification.
Third, if you have checked the above hints, try to fit a cfa only with two of the factors, if the problem persists, try a combination of two other factors and so on.
I don't know your data, but consider using estimator = "MLR" (https://lavaan.ugent.be/tutorial/est.html) since the data is probably skewed (check with describe(data_CFA)

Hope that helps.

Kindly,
Marcus

Franziska Motka

unread,
Apr 9, 2024, 8:29:04 AM4/9/24
to lavaan
Dear Marcus,
thank you, that has already taken me a step further! Now, my model specification looks like this:

### Prepare Data (e.g., standardize and flip to the expected direction)
data_CFA <- subset(data, select=c(AAT_bias_smoking, IAT_app_av_score, IAT_val_score, EMGcorr_score, EMGzygo_score, EMGstartle_score, ACC, lAG, rThal, rStriatum, lSTG))
data_CFA <- as.data.frame(apply(data_CFA,  2, scale)) # apply scale to all columns to centralize and standardize: mean = 0 & SD = 1
# higher scores mean higher sensitization in all measures
data_CFA$EMGcorr_score <- data_CFA$EMGcorr_score * (-1)
data_CFA$EMGstartle_score <- data_CFA$EMGstartle_score * (-1)
data_CFA$lSTG <- data_CFA$lSTG * (-1)

CFAmodel <- '# measurement model
             cognitivebehavioral =~ AAT_bias_smoking + IAT_app_av_score + IAT_val_score

             EMG =~ EMGcorr_score + EMGzygo_score + EMGstartle_score
             fMRI =~ ACC + lAG + rThal + rStriatum + lSTG
             sensitization =~ cognitivebehavioral + EMG + fMRI
             # structural model
             ACC ~~ lAG
             ACC ~~ rThal
             ACC ~~ rStriatum
             ACC ~~ lSTG
             lAG ~~ rThal
             lAG ~~ rStriatum
             lAG ~~ lSTG
             rThal ~~ rStriatum
             rThal ~~ lSTG
             rStriatum ~~ lSTG
             IAT_app_av_score ~~ IAT_val_score
             EMGcorr_score ~~ EMGzygo_score
             EMGcorr_score ~~ EMGstartle_score
             EMGstartle_score ~~ EMGzygo_score

            '
CFAmodel_a <- cfa(CFAmodel, data_CFA, estimator = "MLR", missing = "FIML", std.lv = TRUE) # adressing missing values and skewed data

summary(CFAmodel_a, fit.measures = TRUE, standardized=TRUE)

I added covariances between measures which were collected within the same task, since they should share a certain amount of variance.

However, again, I run into many warning messages... (also without the added covariances)
Warning messages: 1: In lav_model_vcov(lavmodel = lavmodel, lavsamplestats = lavsamplestats, : lavaan WARNING: The variance-covariance matrix of the estimated parameters (vcov) does not appear to be positive definite! The smallest eigenvalue (= -6.731629e-04) is smaller than zero. This may be a symptom that the model is not identified. 2: In lav_object_post_check(object) : lavaan WARNING: some estimated ov variances are negative

The problem is, that simplifying the model (e.g., consider only two factors) would mean that I can not answer my research questions, whether all measures share an underlying mechanism (i.e., sensitization).  

Does anyone has a tip how to deals with this issues?

Best, 
Franziska

Daniel Morillo Cuadrado

unread,
Apr 9, 2024, 9:46:32 AM4/9/24
to lav...@googlegroups.com
You don't need the covariances between measures (in general); they are supposed to be explained by the latent factors (unless some specific observable variables need to be related to one another, beyond what the latent factors explain). The problem with the model underidentification may be due to that.

Refer to my previous message (yesterday) to see how you can deal with your original underidentification.

Best,
--
Daniel Morillo, Ph.D.
GitHub | ORCID


--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/b82e9e37-8ad9-4c25-9f2e-592bd1d234f9n%40googlegroups.com.

Ingo Man

unread,
Apr 9, 2024, 11:15:07 AM4/9/24
to lavaan
Dear Franzi,

underidentification could be a problem indeed. Nevertheless, I would recommend to first specify a model without any restrictions (what you have called "structural model".
How is your sample size? What is the measurement level of your variables? How are missings distributed?

Kindly,
Marcus

Daniel Morillo Cuadrado

unread,
Apr 9, 2024, 11:28:35 AM4/9/24
to lav...@googlegroups.com
It seems like my first message didn't get through (or it doesn't appear in the message list in the Google group at least). My last one may seem cryptic if so... This is what I was saying:

The only part that is (locally) underidentified is the "cognitivebehavioral" latent variable, as it is measured by only two indicators. So you need to add a restriction as you correctly did to one of the parameters, i.e. fixing the loading of AAT_bias_smoking to 1. The rest of the fixed parameters may be adding to many restrictions that the fitted model cannot properly accommodate.

You can try freeing those parameters and leaving only the restriction needed for identification, i.e.:

CFAmodel <- 'cognitivebehavioral =~ 1*AAT_bias_smoking + IAT_app_av_score
             EMG =~ EMGcorr_score + EMGzygo_score + EMGstartle_score
             fMRI =~ ACC + lAG + rThal + rStriatum + lSTG
             sensitization =~ cognitivebehavioral + EMG + fMRI'

Hope that helps!

--
Daniel Morillo, Ph.D.
GitHub | ORCID

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.

Franziska Motka

unread,
Apr 9, 2024, 11:48:46 AM4/9/24
to lavaan
Thank you again for your help!
I ran this model:
CFAmodel <- 'cognitivebehavioral =~ AAT_bias_smoking + IAT_app_av_score + IAT_val_score # in my first post, I forgot to add  IAT_val_score, sorry for confusion

             EMG =~ EMGcorr_score + EMGzygo_score + EMGstartle_score
             fMRI =~ ACC + rThal + rStriatum + lSTG + lAG

             sensitization =~ cognitivebehavioral + EMG + fMRI
            '
CFAmodel_a <- cfa(CFAmodel, data_CFA, estimator = "MLR", missing = "FIML")
And receive the error message:
Warning message: In lavaan::lavaan(model = CFAmodel, data = data_CFA, estimator = "MLR", : lavaan WARNING: the optimizer warns that a solution has NOT been found!

and also:
CFAmodel_a <- cfa(CFAmodel, data_CFA, estimator = "MLR", missing = "FIML", std.lv = TRUE)
Warning messages: 1: In lav_model_vcov(lavmodel = lavmodel, lavsamplestats = lavsamplestats, : lavaan WARNING: Could not compute standard errors! The information matrix could not be inverted. This may be a symptom that the model is not identified. 2: In lav_test_yuan_bentler(lavobject = NULL, lavsamplestats = lavsamplestats, : lavaan WARNING: could not invert information matrix needed for robust test statistic 3: In lav_object_post_check(object) : lavaan WARNING: some estimated ov variances are negative  


Indeed, I have lots of missing data, because the fMRI assessment was only done by a subsample (n=113). Also EMGstartle_score has many missings (observations: n=180). The number of complete cases for all measures is N=60 of a total of N=362. I hope I answered all questions, I also got confused from the messages ;)

Best, Franziska

Daniel Morillo Cuadrado

unread,
Apr 9, 2024, 12:39:16 PM4/9/24
to lav...@googlegroups.com
Ok, I get it now! In that case you don't have identification issues with the model, because all your measurement models are identified.

Then your problem might be due indeed to the missing data. I'd say you might want to try a multiple imputation method, but the number of missing cases is very high indeed.

--
Daniel Morillo, Ph.D.
GitHub | ORCID

Ingo Man

unread,
Apr 10, 2024, 2:35:18 AM4/10/24
to lavaan
Yes, as Daniel mentioned, this could be the problem But nevertheless, try different reduced models, first only include the latent variables that have the least missings, does the model fit? Add further variables. Maybe you could try listwise deletion to see whether or not it fits?

Franziska Motka

unread,
Apr 10, 2024, 8:58:48 AM4/10/24
to lavaan
Thanks again to both of you for your help! This makes perfect sense. I separated the model for simplicity and ran 3 cfa's, one each for cognitive-behavioral, EMG, and fMRI measures:

### COGNITIVE-BEHAVIORAL
CFAmodel_CB <- 'cognitivebehavioral =~  IAT_app_av_score + AAT_bias_smoking + IAT_val_score' 
summary(CFAmodel_CB_output, fit.measures = TRUE, standardized=TRUE)

Warning message: In lav_object_post_check(object) : lavaan WARNING: some estimated lv variances are negative
### EMG
CFAmodel_EMG <- 'EMG =~ EMGstartle_score + EMGzygo_score + EMGcorr_score'
CFAmodel_EMG_output <- cfa(CFAmodel_EMG, data_CFA, missing = "FIML")
summary(CFAmodel_EMG_output, fit.measures = TRUE, standardized=TRUE)

Warning message: In lav_object_post_check(object) : lavaan WARNING: some estimated lv variances are negative
### fMRI
CFAmodel_fMRI <- 'fMRI =~ ACC + lAG + rThal + rStriatum + lSTG'
CFAmodel_fMRI_output <- cfa(CFAmodel_fMRI, data_CFA, missing = "FIML")#, std.lv = TRUE
summary(CFAmodel_fMRI_output, fit.measures = TRUE, standardized=TRUE) 
--> works!

The error messages for the models of the cognitive-behavioral and EMG measures also occur when using only complete cases (cognitive-behavioral: n = 348, EMG: n = 173). I read some common reasons for this problem (e.g. underidentification, missing data, manifest variables have different scales, skewed data), but I think I addressed all these possibilities (I standardized the data, used only complete cases, tried all robust estimators on complete case analysis, and the model should not be underidentified). So if you have an idea, let me know.

However, I think I have found a reasonable solution, unless I have misunderstood something. I have specified this model:
CFAmodel_full <- 'cognitivebehavioral =~ 1/3*AAT_bias_smoking + 1/3*IAT_app_av_score + 1/3*IAT_val_score
             EMG =~ 1/3*EMGcorr_score + 1/3*EMGzygo_score + 1/3*EMGstartle_score
             fMRI =~ 1/5*ACC + 1/5*rThal + 1/5*rStriatum + 1/5*lSTG + 1/5*lAG

             sensitization =~ cognitivebehavioral + EMG + fMRI'
CFAmodel_full_output <- cfa(CFAmodel_full, data_CFA, missing = "FIML", std.lv = TRUE)
summary(CFAmodel_full_output, fit.measures = TRUE, standardized=TRUE) 

This works without an error message, and I think it answers my research question of whether there is a common mechanism among all 11 measures separated at the cognitive-behavioral, psychophysical (EMG), and neural (fMRI) level. I assign equal loadings to the manifest variables because I am not interested in those loadings anyway. I am only interested in the estimates between the first and second order latent variables. I also think the results make sense. All of the fMRI measures are moderately correlated (as I know from simple correlational analyses), whereas the EMG and cognitive-behavioral measures are not. Thus, the fMRI measures share a large amount of variance, and therefore the second order variable "contains" this variance, leading to the highest estimate. 
Latent Variables: Estimate Std.Err z-value P(>|z|) Std.lv Std.all sensitization =~
cognitivebhvrl -0.208 0.166 -1.248 0.212 -0.203 -0.203 EMG -0.157 0.201 -0.781 0.435 -0.155 -0.155 fMRI 3.469 0.308 11.259 0.000 0.961 0.961

If you have any concerns about this solution, please let me know. 
Best, 
Franziska

Ingo Man

unread,
Apr 11, 2024, 4:14:04 AM4/11/24
to lavaan
Hey,

  • "The error messages for the models of the cognitive-behavioral and EMG measures also occur when using only complete cases (cognitive-behavioral: n = 348, EMG: n = 173). I read some common reasons for this problem (e.g. underidentification, missing data, manifest variables have different scales, skewed data), but I think I addressed all these possibilities (I standardized the data, used only complete cases, tried all robust estimators on complete case analysis, and the model should not be underidentified). So if you have an idea, let me know."
Nice to see that you have tried a lot of different options. Have you tried using (a) different marker indicator(s)?
It is very It is very strange that the first two models are not running...

If I got it correctly, you designed a higher order factor model, right?

The approach to assign equal factor loadings seems plausible. You did it via assigning the same label (i. e. "1/3"), my first thought was, this would be kind of a formula to spread the loadings equally among the indicators, but giving a label such as "v1" for each of the indicators of a factor should do the same.
What I would also try, as a sensitivity analyses, would be to play around with the factor loadings; know you assume that they are the same within each of the factors; maybe try to specify different ratios of loading distributions and compare the results. I am not to deep into this, but if your results are stable, than everything should be fine.

Kindly,
Marcus

Daniel Morillo Cuadrado

unread,
Apr 12, 2024, 9:10:13 AM4/12/24
to lav...@googlegroups.com
The only part that is (locally) underidentified is the "cognitivebehavioral" latent variable, as it is measured by only two indicators. So you need to add a restriction as you correctly did to one of the parameters, i.e. fixing the loading of AAT_bias_smoking to 1. The rest of the fixed parameters may be adding to many restrictions that the fitted model cannot properly accommodate.

You can try freeing those parameters and leaving only the restriction needed for identification, i.e.:

CFAmodel <- 'cognitivebehavioral =~ 1*AAT_bias_smoking + IAT_app_av_score
             EMG =~ EMGcorr_score + EMGzygo_score + EMGstartle_score
             fMRI =~ ACC + lAG + rThal + rStriatum + lSTG
             sensitization =~ cognitivebehavioral + EMG + fMRI'

Hope that helps!
--
Daniel Morillo, Ph.D.
GitHub | ORCID

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.

Jošt Bartol

unread,
Apr 12, 2024, 9:10:20 AM4/12/24
to lav...@googlegroups.com
Hi Franziska,

The model is correctly specified and should be identified. However, there might be other issues such as small samples size, problems in data processing, severe model misspecification, problems in relationships between manifest variables etc. I suggest you review additional model information such as: size of factor loadings, residuals, distributions and correlations among manifest variables, check if all data were correctly prepared (i.e., no errors in data preparation). This can probably help you identify the issue and try  to remedy it; maybe by dropping some items. In sum, I don't think a precise answer can be given, you need to do a diagnostic check.

Note that your second specification (with some loadings fixed to 1) is not appropriate since you are already setting latent variances to 1 (i.e., "std.lv = TRUE" argument). Either set latent variances to 1 ("std.lv = TRUE") or drop this argument and fix loadings to 1 (although this is what lavaan does by default; if you don't specify std.lv = TRUE of course).

Best,
Jošt

V V pon., 8. apr. 2024 ob 15:39 je oseba Franziska Motka <franzis...@gmail.com> napisala:
--

Franziska Motka

unread,
Apr 12, 2024, 10:42:20 AM4/12/24
to lavaan
Hi Marcus,

yes, I also tried all possible marker indicators. I guess the reason is that all cognitive-behavioral and all EMG measures do not correlate at all, while all fMRI measures do correlate. So I "force" the cfa function to calculate estimates for a model representing their association, but in data reality, they are not associated at all... On the other hand, this is exactly what we expect, because we think that the theory is not appropriate, at least in our sample. 

Yes, from a theoretical point, this model makes most sense (thank you for the hint with the parameters instead of fractions):
CFAmodel_full <- 'cognitivebehavioral =~ a*AAT_bias_smoking + a*IAT_app_av_score + a*IAT_val_score
             EMG =~ b*EMGcorr_score + b*EMGzygo_score + b*EMGstartle_score
             fMRI =~ c*ACC + c*rThal + c*rStriatum + c*lSTG + c*lAG

             sensitization =~ cognitivebehavioral + EMG + fMRI
            '
Thank you very much for your help! 
I think I found a "presentable" solution ;)

Ingo Man

unread,
Apr 15, 2024, 4:50:33 AM4/15/24
to lavaan
Hello again,

Jošt has taken the discussion precisely in the desired direction. Since your variables per factor seem to not correlate very much (or at all), this alone should be sufficient justification as to why the final model does not fit/does not converge. Factor loads should always be inspected.

Good luck.
Marcus
Reply all
Reply to author
Forward
0 new messages