vcov not positive definite warning in CFA with 5-point Likert-scale items using WLSMV
김지연
Hello,
I am conducting a scale-development study and have a question about a recurring vcov warning in CFA using lavaan.
I fitted a 25-item, 5-factor CFA (N = 200) with 5-point Likert-scale items. I specified the items as ordered and used WLSMV:
fit <- cfa(model, data = CFA200, estimator = "WLSMV", ordered = items)
The model converged, but I consistently received the following warning:
The variance-covariance matrix of the estimated parameters (vcov) does not appear to be positive definite! The smallest eigenvalue (= -7.746245e-18) is smaller than zero. This may be a symptom that the model is not identified.
For this 25-item model, I checked the following:
- converged = TRUE
- post.check = TRUE
- standardized factor loadings ranged from .538 to .950
- all residual variances were positive
- no Heywood cases were observed
- latent factor correlations ranged from .338 to .811
One of the five factors contains only two indicators, so I initially suspected that this might be related to the warning. However, after removing this factor entirely and fitting the remaining 23-item, 4-factor model, the same type of vcov warning remained, with the smallest eigenvalue again extremely close to zero (3.59e-18).
I also tested a separate candidate factor structure derived from the EFA: a 28-item, 5-factor model using the same independent CFA sample (N = 200). In this model, all five factors had at least three indicators (13, 4, 4, 4, and 3 indicators). Nevertheless, WLSMV produced the same warning:
The variance-covariance matrix of the estimated parameters (vcov) does not appear to be positive definite! The smallest eigenvalue (= -2.443949e-17) is smaller than zero. This may be a symptom that the model is not identified.
The 28-item model also converged normally (converged = TRUE, post.check = TRUE), and the standardized factor loadings ranged from .525 to .897, with no loadings greater than 1.
As an additional diagnostic check, I fitted the original 25-item, 5-factor structure using the default ML estimator, treating the items as continuous. In this case, the vcov warning did not occur. However, when I used sem() instead of cfa() and again specified estimator = "WLSMV" and ordered = items, the same warning reappeared, with exactly the same smallest eigenvalue (-7.746245e-18).
Therefore, the warning appears repeatedly when I specify the 5-point Likert-scale items as ordered and use WLSMV, and it persists even after removing the two-indicator factor and when testing another factor structure in which all factors have at least three indicators.
I would greatly appreciate your advice on the following questions:
- Given that the models converge, post.check = TRUE, no Heywood cases are observed, and the smallest eigenvalues are extremely close to zero (approximately 10^-18 to 10^-17), could these warnings reflect numerical near-singularity or numerical precision rather than substantive model non-identification?
- Are there any additional diagnostics you would recommend before interpreting and reporting the WLSMV CFA results?
- In the 25-item model, should the two-indicator factor be considered a separate identification concern even though removing the entire factor did not eliminate the vcov warning?
I am using lavaan version 0.7-2.
Thank you very much for your time and guidance.
Edward Rigdon
issue. It may be making parameter estimates unstable and also making
the vcov estimate itself unstable. You might consider simulating a
larger sample with the same summary statistics as your n = 200 sample
and seeing whether the message persists.
With only n = 200 and 5 response categories per item, your
responses will be distributed across 5 x 5 grids for each pair of
observed variables. It may be that there are pairs of variables with a
large number of empty cells--or maybe a few observed variables with
"wild card" observations that don't conform to the overall pattern..
That would certainly contribute to instability. Looking at 2 x 2
crosstabs will be tedious, but it is always a good idea to look at the
data.
To better understand where the issue lies, I was going to suggest
that you request the correlation matrix of the parameter estimates,
but I don't see such a keyword in the R manual. (You
could--tediously--request the vcov and standardize it yourself.) You
would be looking for correlations *very* close to +/-1, because values
beyond +/- 0.9 can occur even in stable solutions. In addition,
near-singularity can happen when no single correlation is extremely
large but a pattern of large correlations exists. Still, you may find
that teh instability revolves around what is essentially a nuisance
parameter. If you can report that to a reader or reviewer, that would
support the case that the warning is not a major concern.
Knowing which specific parameter estimates are involved would
also help you to invoke another checking strategy--specifying various
starting values and then seeing whether the model converges to the
same solution, or deviates.
> You received this message because you are subscribed to the Google Groups "lavaan" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/lavaan/33d98bc1-3bd4-40b9-8cde-ff520ef5cf67n%40googlegroups.com.
김지연
Thank you very much for your detailed and helpful response.
Based on your suggestions, I performed two additional checks on the 28-item, 5-factor model (N = 200) that produced the same vcov warning.
First, I obtained the variance-covariance matrix of the parameter estimates (vcov) and standardized it to examine the correlations among the parameter estimates. There were five pairs with |r| > .95, and all of them involved only the first thresholds (t1). Four of these correlations were 1.000000000000000, and the remaining one was 0.999999999999999. In contrast, I found no correlations above |.95| involving factor loadings or factor covariances.
I also examined the response distributions of the items involved in these nearly perfect correlations between the first thresholds. For each of these items, only 1–2 out of 200 participants (0.5–1.0%) selected response category 1.
Second, I examined the empty cells in the 5 × 5 pairwise response tables, as you suggested. Among the 378 possible item pairs for the 28 items, 377 pairs had at least one empty cell. The mean number of empty cells per item pair was 8.35 (median = 8.5, range = 0–15).
I would like to make sure that I am interpreting these findings correctly. Would it be reasonable to view this as the situation you described in which the instability is concentrated in nuisance parameters, such as thresholds, rather than in substantive measurement-model parameters such as factor loadings or factor relationships?
Also, considering together the nearly perfect correlations among the first-threshold estimates, the sparsity of the lowest response category, and the large number of empty cells in the 5 × 5 pairwise response tables, would it be reasonable to interpret the vcov warning as possibly reflecting numerical instability associated with sparse ordinal response distributions rather than a substantive model non-identification problem?
If so, I would greatly appreciate your advice on whether these diagnostic findings provide sufficient support for considering the warning unlikely to represent a major problem, or whether further diagnostic checks would still be warranted.
Thank you again for your time and helpful guidance.
Edward Rigdon
hoped for. I agree with your interpretation. I think you can complete
the case for your interpretation by collapsing the first and second
response categories (recode the single "1," if you like, as
"2"--remember that the specific numerical codes are irrelevant, except
for indicating order) and re-running the analysis. If this new
analysis produces essentially the same findings as the original--and
without the vcov warning--then I think you can continue with the
interpretation that you describe.
For the benefit of future researchers who may use this scale (and
encounter the same warning) and for reviewers (who may seek to
reproduce your analysis during the review process) and editors, it
will be good to include a brief description of your diagnostic efforts
and results in a footnote explaining why you are moving past the
warning message and carrying on with your interpretation.
The core issue, I think, is sample size. Textbooks and other
literature from the 1980s says that n = 200 is sufficient for factor
analysis. This was poor general advice, and never well-defended, but
it was popular. Today, especially with texotic estimators (like WLS
variants) in common use, and with ordinal data in particular, we need
to think more carefully about sample size. There probably is ample
room for a paper in a good journal that updates and defends an
evidence-based approach for setting necessary sample size in CFA.
Good luck with your research.