CFA problems with latent variables

Leonie Krol

unread,

Mar 9, 2023, 10:21:04 AM3/9/23

to lavaan

Dear lavaan community,

I want to test the factorial validity of an existing questionnaire in a clinical sample. My problem is that I get a warning message that I don’t understand. I have checked many times that I’m using the correct data frames etc. and still can’t find the problem. I have a clinical sample with 127 complete observations, the questionnaire has 17 items answered on a 3-point Likert-type scale. Below is the detailed information.

I used the following model and the cfa-funtion to fit it:

eight.model <- ' # measurement model

physical =~ pcsisr801 +

pcsisr802 +

pcsisr803 +

pcsisr804 +

pcsisr805 +

pcsisr806 +

pcsisr807 +

pcsisr808

emotional =~ pcsisr809 +

pcsisr810 +

pcsisr811

cognitive =~ pcsisr812 +

pcsisr813 +

pcsisr814 +

pcsisr815

fatigue =~ pcsisr816 +

pcsisr817

'

fit.pcsi8.o <- cfa(eight.model, data = pcsi8.2, ordered = items.pcsi8, std.lv = T)

Here is the warning message:

Warning message:

In lav_object_post_check(object) :

lavaan WARNING: covariance matrix of latent variables

is not positive definite;

use lavInspect(fit, "cov.lv") to investigate.

If I check with lavInspect(fit.pcsi8.o, "cov.lv"), I get this output:

physcl emotnl cogntv fatigu

physical 1.000

emotional 0.824 1.000

cognitive 0.999 0.784 1.000

fatigue 0.900 0.607 0.844 1.000

Unfortunately, I don't understand what the problem is. All standardized covariances (i.e., correlations) are smaller than 1 and positive.

Even when I check with summary(fit.pcsi8.o) I can’t find any problematic covariances, neither in the latent variables nor in any other parameter. Here is the output:

lavaan 0.6.14 ended normally after 28 iterations

Estimator DWLS

Optimization method NLMINB

Number of model parameters 57

Number of observations 127

Model Test User Model:

Standard Scaled

Test Statistic 66.916 116.764

Degrees of freedom 113 113

P-value (Chi-square) 1.000 0.385

Scaling correction factor 1.002

Shift parameter 49.983

simple second-order correction

Parameter Estimates:

Standard errors Robust.sem

Information Expected

Information saturated (h1) model Unstructured

Latent Variables:

Estimate Std.Err z-value P(>|z|)

physical =~

pcsisr801 0.522 0.082 6.385 0.000

pcsisr802 0.758 0.082 9.258 0.000

pcsisr803 0.756 0.081 9.302 0.000

pcsisr804 0.751 0.063 11.901 0.000

pcsisr805 0.763 0.071 10.676 0.000

pcsisr806 0.774 0.061 12.779 0.000

pcsisr807 0.605 0.085 7.078 0.000

pcsisr808 0.719 0.061 11.702 0.000

emotional =~

pcsisr809 0.760 0.068 11.193 0.000

pcsisr810 0.659 0.088 7.468 0.000

pcsisr811 0.882 0.051 17.233 0.000

cognitive =~

pcsisr812 0.812 0.055 14.703 0.000

pcsisr813 0.621 0.084 7.358 0.000

pcsisr814 0.685 0.079 8.643 0.000

pcsisr815 0.904 0.048 18.808 0.000

fatigue =~

pcsisr816 0.838 0.057 14.640 0.000

pcsisr817 0.912 0.063 14.563 0.000

Covariances:

Estimate Std.Err z-value P(>|z|)

physical ~~

emotional 0.824 0.076 10.906 0.000

cognitive 0.999 0.039 25.754 0.000

fatigue 0.900 0.052 17.167 0.000

emotional ~~

cognitive 0.784 0.084 9.290 0.000

fatigue 0.607 0.119 5.122 0.000

cognitive ~~

fatigue 0.844 0.070 12.147 0.000

Intercepts:

Estimate Std.Err z-value P(>|z|)

.pcsisr801 0.000

.pcsisr802 0.000

.pcsisr803 0.000

.pcsisr804 0.000

.pcsisr805 0.000

.pcsisr806 0.000

.pcsisr807 0.000

.pcsisr808 0.000

.pcsisr809 0.000

.pcsisr810 0.000

.pcsisr811 0.000

.pcsisr812 0.000

.pcsisr813 0.000

.pcsisr814 0.000

.pcsisr815 0.000

.pcsisr816 0.000

.pcsisr817 0.000

physical 0.000

emotional 0.000

cognitive 0.000

fatigue 0.000

Thresholds:

Estimate Std.Err z-value P(>|z|)

pcsisr801|t1 0.270 0.113 2.384 0.017

pcsisr801|t2 1.530 0.175 8.748 0.000

pcsisr802|t1 0.771 0.125 6.186 0.000

pcsisr802|t2 1.530 0.175 8.748 0.000

pcsisr803|t1 0.973 0.133 7.306 0.000

pcsisr803|t2 1.758 0.204 8.632 0.000

pcsisr804|t1 0.853 0.128 6.677 0.000

pcsisr804|t2 2.151 0.281 7.657 0.000

pcsisr805|t1 1.005 0.135 7.456 0.000

pcsisr805|t2 1.984 0.243 8.172 0.000

pcsisr806|t1 1.038 0.137 7.604 0.000

pcsisr806|t2 2.151 0.281 7.657 0.000

pcsisr807|t1 0.719 0.123 5.852 0.000

pcsisr807|t2 1.984 0.243 8.172 0.000

pcsisr808|t1 0.573 0.119 4.829 0.000

pcsisr808|t2 1.672 0.192 8.720 0.000

pcsisr809|t1 -0.010 0.112 -0.088 0.930

pcsisr809|t2 1.672 0.192 8.720 0.000

pcsisr810|t1 0.395 0.115 3.438 0.001

pcsisr810|t2 1.597 0.182 8.754 0.000

pcsisr811|t1 0.596 0.119 5.001 0.000

pcsisr811|t2 1.597 0.182 8.754 0.000

pcsisr812|t1 0.668 0.121 5.514 0.000

pcsisr812|t2 1.859 0.220 8.465 0.000

pcsisr813|t1 0.353 0.114 3.087 0.002

pcsisr813|t2 1.758 0.204 8.632 0.000

pcsisr814|t1 0.482 0.116 4.136 0.000

pcsisr814|t2 1.859 0.220 8.465 0.000

pcsisr815|t1 0.882 0.129 6.837 0.000

pcsisr815|t2 1.672 0.192 8.720 0.000

pcsisr816|t1 0.668 0.121 5.514 0.000

pcsisr816|t2 1.597 0.182 8.754 0.000

pcsisr817|t1 0.853 0.128 6.677 0.000

pcsisr817|t2 1.672 0.192 8.720 0.000

Variances:

Estimate Std.Err z-value P(>|z|)

.pcsisr801 0.727

.pcsisr802 0.425

.pcsisr803 0.428

.pcsisr804 0.437

.pcsisr805 0.418

.pcsisr806 0.401

.pcsisr807 0.634

.pcsisr808 0.483

.pcsisr809 0.422

.pcsisr810 0.565

.pcsisr811 0.223

.pcsisr812 0.341

.pcsisr813 0.615

.pcsisr814 0.531

.pcsisr815 0.183

.pcsisr816 0.298

.pcsisr817 0.169

physical 1.000

emotional 1.000

cognitive 1.000

fatigue 1.000

Scales y*:

Estimate Std.Err z-value P(>|z|)

pcsisr801 1.000

pcsisr802 1.000

pcsisr803 1.000

pcsisr804 1.000

pcsisr805 1.000

pcsisr806 1.000

pcsisr807 1.000

pcsisr808 1.000

pcsisr809 1.000

pcsisr810 1.000

pcsisr811 1.000

pcsisr812 1.000

pcsisr813 1.000

pcsisr814 1.000

pcsisr815 1.000

pcsisr816 1.000

pcsisr817 1.000

Can you please help me understand the latent variable warning message of and give me some advice on how to solve this problem?

Thank you and best wishes,

Leonie

Edward Rigdon

unread,

Mar 9, 2023, 11:17:39 AM3/9/23

to lav...@googlegroups.com

Leonie--

The warning message--not an error--is actually pretty straightforward. If a set of variables are not redundant, then their covariance matrix will have a positive determinant. Based on your model, the covariance matrix of your common factors does not. This can happen is a variance is negative or if two or more variables have correlations or multiple correlations that are too high--or if the factor model is simply wrong for the data, leading to "impossible" parameter estimates. "Too high" is the immediate issue with your results.

In the correlation matrix of common factors, you have a correlation (between factors "cognitive" and "physical") estimated as 0.999, so effectively 1. Keep in mind that the conclusion that a matrix has a positive determinant is affected by random sampling variance, so that the very same matrix may be judged "OK" if sample size is large but "not OK" if sample size is small. But a correlation of 0.999 is likely to produce this warning regardless of sample size. Even the 0.999 correlation between the "physical" and "fatigue" factors may be high enough to trigger the warning, at low sample size..

Keep in mind that you are estimating 57 model parameters using 127 cases, so there may be some instability in your parameter estimates, beyond what is reflected in standard errors.

Model fit--in the sense of discrepancy between model and data--appears to be good. But the 0.999 correlation suggests that the "physical" and "cognitive" factors are not distinct. To see this more clearly, you might estimate a model where those items all load on a single common factor. If you were hypothesizing that a model with distinct factors was better for your population, these results do not tend to support that position.

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/6ee56a73-efa2-4d5c-bb60-07a03c54901cn%40googlegroups.com.

Shu Fai Cheung

unread,

Mar 9, 2023, 7:26:48 PM3/9/23

to lav...@googlegroups.com

It is a clinical sample and the items only have three options. Maybe the problem occurred at item level, with some items extremely highly correlated?

You used "ordered". You can lavInspect(), e.g., lavInspect(fit.pcsi8.o, "sampstat") and examine the correlations (estimated from the contingency tables), and then examine the 3x3 contingency tables of item-pairs with high correlations.

My two cents.

Regards,

Shu Fai Cheung (張樹輝)

--

Leonie Krol

unread,

May 10, 2023, 5:05:16 AM5/10/23

to lavaan

Dear all,

I apologise for my late reply.

Thank you for your feedback! It helped me a lot.

Best regards,
Leonie

cla...@googlemail.com

unread,

Jul 4, 2023, 4:18:43 AM7/4/23

to lavaan

Hi there,

I am also running a cfa with ordered categorical indicators (0-2, 3-point-likert, 5 latent factors, each 5 indicators). My sample size is sufficient (N= 13.000).

My problem is, that when inspecting this simple model, I do find negative values for thresholds (standardized and non-standardized). Which seems odd. Lavaan automatically fixes a few parameters as usual, mostly due to the parameterization command. Later on in my analysis, I am working with measEq.syntax using the Wu & Esbrook 2016 specification, since I am testing Measurement Invariance. However, I wanted to check if the simple model before these changes has a sufficient good fit and could not find any information on how to evaluate the freely estimated thresholds. Do I have to be concerned, and if, how can I investigate or rather fix this? Note, my scale is 0 to 2 in three ordered catgories. The freely estimated threshold values below seem to be partly outside the realm of the observed values.

Here, my code:

library(lavaan)
mod.cat <- '

pemotion =~ n_pworries + n_psomatic + n_punhappy + n_pclingy + n_pafraid

pconduct =~ n_qobeys + n_ptantrum + n_pfights + n_plies + n_psteals

phyper =~ n_prestles + n_pfidgety + n_pdistrac + n_qreflect + n_qattends

ppeer =~ n_qpopular + n_ploner + n_qfriend +n_pbullied + n_poldbest

pprosoc =~ n_phelpout + n_pconsid + n_pshares + n_pcaring + n_pkind
# fist indicators are MARKER items
'

fit.cat <- cfa(mod.cat, data = df_mi,
ordered = c("n_psomatic", "n_pworries", "n_punhappy", "n_pclingy", "n_pafraid",
"n_ptantrum", "n_qobeys", "n_pfights", "n_plies", "n_psteals",
"n_prestles", "n_pfidgety", "n_pdistrac", "n_qreflect", "n_qattends",
"n_ploner", "n_qfriend", "n_qpopular", "n_pbullied", "n_poldbest",
"n_pconsid", "n_pshares", "n_pcaring", "n_pkind", "n_phelpout"),

estimator ="DWLS",
test = "scaled.shifted",
se = "robust",
meanstructure = TRUE,
parameterization = "theta")

Here are my thresholds:

Thresholds: Estimate Std.Err z-value P(>|z|) Std.lv Std.all n_pworries|t1 1.283 0.035 36.899 0.000 1.283 0.823 n_pworries|t2 3.079 0.071 43.653 0.000 3.079 1.975 n_psomatic|t1 0.571 0.015 38.508 0.000 0.571 0.532 n_psomatic|t2 1.836 0.026 71.035 0.000 1.836 1.711 n_punhappy|t1 1.682 0.052 32.606 0.000 1.682 1.025 n_punhappy|t2 3.291 0.090 36.732 0.000 3.291 2.005 n_pclingy|t1 0.125 0.018 7.066 0.000 0.125 0.091 n_pclingy|t2 1.855 0.033 55.594 0.000 1.855 1.349 n_pafraid|t1 0.620 0.020 31.123 0.000 0.620 0.456 n_pafraid|t2 2.296 0.039 58.702 0.000 2.296 1.689 n_qobeys|t1 -0.301 0.016 -19.154 0.000 -0.301 -0.248 n_qobeys|t2 2.127 0.032 65.945 0.000 2.127 1.755 n_ptantrum|t1 0.063 0.016 3.905 0.000 0.063 0.050 n_ptantrum|t2 1.748 0.027 65.816 0.000 1.748 1.390 n_pfights|t1 1.123 0.025 45.014 0.000 1.123 0.845 n_pfights|t2 2.829 0.053 53.172 0.000 2.829 2.128 n_plies|t1 0.457 0.017 27.465 0.000 0.457 0.373 n_plies|t2 2.233 0.034 65.970 0.000 2.233 1.823 n_psteals|t1 1.976 0.036 55.325 0.000 1.976 1.782 n_psteals|t2 2.506 0.048 52.135 0.000 2.506 2.260 n_prestles|t1 0.492 0.026 18.642 0.000 0.492 0.262 n_prestles|t2 2.556 0.052 48.871 0.000 2.556 1.361 n_pfidgety|t1 0.880 0.030 29.078 0.000 0.880 0.474 n_pfidgety|t2 2.913 0.059 49.404 0.000 2.913 1.569 n_pdistrac|t1 -0.407 0.021 -19.055 0.000 -0.407 -0.245 n_pdistrac|t2 1.840 0.033 56.252 0.000 1.840 1.107 n_qreflect|t1 -0.570 0.018 -32.065 0.000 -0.570 -0.433 n_qreflect|t2 2.202 0.034 65.680 0.000 2.202 1.671 n_qattends|t1 -0.409 0.021 -19.770 0.000 -0.409 -0.257 n_qattends|t2 2.202 0.037 59.485 0.000 2.202 1.386 n_qpopular|t1 1.107 0.036 30.407 0.000 1.107 0.669 n_qpopular|t2 3.629 0.100 36.138 0.000 3.629 2.192 n_ploner|t1 0.755 0.019 40.663 0.000 0.755 0.624 n_ploner|t2 2.153 0.035 61.409 0.000 2.153 1.780 n_qfriend|t1 1.474 0.031 47.321 0.000 1.474 1.168 n_qfriend|t2 2.443 0.048 51.364 0.000 2.443 1.937 n_pbullied|t1 1.279 0.033 38.841 0.000 1.279 0.904 n_pbullied|t2 2.908 0.065 44.570 0.000 2.908 2.054 n_poldbest|t1 0.768 0.018 43.240 0.000 0.768 0.662 n_poldbest|t2 2.073 0.033 63.666 0.000 2.073 1.785 n_phelpout|t1 -2.214 0.034 -64.343 0.000 -2.214 -1.795 n_phelpout|t2 0.055 0.016 3.474 0.001 0.055 0.044 n_pconsid|t1 -3.375 0.098 -34.274 0.000 -3.375 -1.920 n_pconsid|t2 0.091 0.022 4.071 0.000 0.091 0.052 n_pshares|t1 -2.380 0.038 -61.931 0.000 -2.380 -1.922 n_pshares|t2 -0.235 0.016 -14.521 0.000 -0.235 -0.189 n_pcaring|t1 -2.797 0.053 -52.737 0.000 -2.797 -2.081 n_pcaring|t2 -1.085 0.025 -42.943 0.000 -1.085 -0.807 n_pkind|t1 -2.416 0.040 -60.554 0.000 -2.416 -2.059 n_pkind|t2 -1.048 0.020 -51.472 0.000 -1.048 -0.893

I am happy to share more information. Thanks for all the help in advance. I appreciate your input in advance.

Best

Claudia

Yago Luksevicius de Moraes

unread,

Jul 5, 2023, 11:15:33 AM7/5/23

to lavaan

Hi Claudia,

Not sure what you mean, but you do have a few thresholds in the negative range of the latent scale (e.g., n_pkind|t1, n_pcaring|t1).

Further, the threshold is the percentile in a standardized normal distribution that represents the accumulated frequency of respondents that chose a category k or lower. So, this seems to suggest that your data is very skewed (i.e., almost all answers are in one extreme of the item).

It is not a problem for factorial validation per se, as far as I know, but it suggests that your items have ceiling effects, i.e., it is to easy to strongly (dis)agree in this population and a high chance of getting no respondents in the opposite extreme (from what I could guess about your items, those in the opposite extreme may be considered as needing professional interventions, and the "typical" respondents are "just normal". If this is so, I would hypothesize the scale is good for this subsample, but not for normal individuals. Maybe you can divide your sample in two and compare them).

Best,

Yago

cla...@googlemail.com

unread,

Jul 17, 2023, 10:01:21 AM7/17/23

to lavaan

Hello Yago,

thank you for your input, this was valuable information and I think it is right on point and also gives me some ideas how to tackle the problem. In my case, this is indeed a screener for potential psychological problems and it makes a lot of sense if it separates potential clinical samples from others. It should work better for this target group. However, I will check if I can indeed do a validation in two separate samples (population-based and clincial sample).

Thanks again and best of luck!

Claudia

Yago Luksevicius de Moraes

unread,

Jul 17, 2023, 1:34:40 PM7/17/23

to lav...@googlegroups.com

Glad it helped.

Good luck,

Yago

You received this message because you are subscribed to a topic in the Google Groups "lavaan" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lavaan/eUH4PHfuQjQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/bbfcbfc9-fca4-4d9c-83e6-5636ec6a3f1an%40googlegroups.com.

--

Yago Luksevicius de Moraes

Master in Experimental Psychology

Bachelor in Psychology

http://lattes.cnpq.br/5324856209740351

https://orcid.org/0000-0002-7802-2002

Reply all

Reply to author

Forward