Suggestion: extend check.post to the parameter matrices

[adri...@gmail.com]

A structural model I fitted in lavaan 0.7-2 (WLSMV, ordered indicators) converged normally, issued no warnings, and returned TRUE for lavInspect(fit, "post.check"). The solution was nevertheless improper: psi had an eigenvalue of -0.269, because the freely estimated residual covariances among three single-indicator endogenous outcomes were far larger than their residual variances allow, giving residual correlations of -3.55 and -2.28.

This is documented behaviour, not a bug. check.post verifies the variances and the model-implied covariance matrices, and both of those were fine here: the smallest eigenvalues of cov.lv and cov.ov were 0.285 and 0.133.

Would it be worth extending check.post, or adding a separate warning, to psi and theta? The impossible values are already visible in the Std.all column of summary(), but among sixty other numbers they are easy to read past when a model has otherwise converged cleanly.

ATB

Adriaan

[Stas Kolenikov]

No. Algorithmic convergence and model validity are two separate issues. Checking whether the algorithm converged is  the question of “do we need to iterate any further?” and it is answered, roughly speaking, by how close the empirical covariance matrix and the implied covariance matrix are. If we tried to measure convergence by whether all parameters are meaningful, we will never get convergence in your case. 

-- Stas Kolenikov, PhD, PStat (ASA, SSC) 

-- Principal Statistician, NORC @NORCnews

-- Opinions stated in this email are mine only, and do not reflect the position of my employer

-- Social media: @StatStas [ Twitter | mastodon.online ]

-- http://stas.kolenikov.name

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/lavaan/4d4949a0-c164-4efd-a409-6c916da907b1n%40googlegroups.com.

[adri...@gmail.com]

Thanks. Agreed on convergence. My question was about check.post, which runs after convergence and only issues a warning. Its documentation describes it as a post-fitting check on whether the solution is admissible, covering non-negative variances and positive semi-definiteness of the model-implied covariance matrices. My suggestion was to include psi and theta in that same post-fitting warning, leaving the convergence criterion untouched. Would that be worth considering?

[Felipe Vieira]

Hi Adriaan, 

I checked the source code (R/lav_object_post_check.R, if curious) and I believe your request makes sense. I still need to double check it, but there seems to be something going on with observed endogenous variables being treated as dummy latent variables. It would help if you could share a reprex though (?). 

I will open an issue on GitHub in the meantime. 

Best, 

Felipe. 

[adri...@gmail.com]

Hi Felipe,

Thanks for looking at the source and for opening the issue.

Your dummy latent variable idea matches what I see. In my own fit the residual covariances of the three ordinal endogenous outcomes appear in both psi and theta, and it is the psi block that fails: eigenvalues .407, .169 and -0.269, with residual correlations of -3.55 and -2.28. The psi diagonal is positive throughout, the smallest being .048, so nothing shows up there and post.check returns TRUE.

The original data cannot be shared. Below is a simulated example with the same structure as the model where I met this: four correlated factors on ordinal indicators, a genuine method component in the items, three single-indicator ordinal outcomes whose residual covariances sem() frees, and an equality-constrained unmeasured latent method factor orthogonal to the substantive factors, as in Podsakoff et al. (2003).

The reprex (drafted with an AI assistant, then checked and corrected):

r

# lavaan 0.7-2: post.check returns TRUE while psi is not positive definite. # The psi diagonal stays positive; the matrix fails through its off-diagonal. library(lavaan) set.seed(1) n <- 500 fcor <- 0.85 # correlation among the four substantive factors gamma <- 0.55 # method component in each item e_sd <- 0.60 # noise on the outcomes cut5 <- function(v) as.numeric(cut(v, c(-Inf, -1, -0.3, 0.3, 1, Inf), labels = FALSE)) L <- chol(matrix(fcor, 4, 4) + diag(1 - fcor, 4)) Z <- matrix(rnorm(n * 4), n, 4) %*% L # true method component in the data, distinct from the CMV factor fitted below m <- rnorm(n) lam <- rep(c(.85, .80, .80), 4) d <- data.frame(matrix(NA, n, 12)) names(d) <- paste0("x", 1:12) for (j in 1:12) { eta <- Z[, ceiling(j / 3)] d[[j]] <- cut5(lam[j] * eta + gamma * m + rnorm(n, sd = sqrt(max(1 - lam[j]^2 - gamma^2, .05)))) } for (k in 1:3) d[[paste0("y", k)]] <- cut5(0.75 * rowMeans(Z) + rnorm(n, sd = e_sd)) mod <- ' F1 =~ x1 + x2 + x3 F2 =~ x4 + x5 + x6 F3 =~ x7 + x8 + x9 F4 =~ x10 + x11 + x12 y1 + y2 + y3 ~ F1 + F2 + F3 + F4 CMV =~ a*x1 + a*x2 + a*x3 + a*x4 + a*x5 + a*x6 + a*x7 + a*x8 + a*x9 + a*x10 + a*x11 + a*x12 CMV ~~ 1*CMV F1 ~~ 0*CMV F2 ~~ 0*CMV F3 ~~ 0*CMV F4 ~~ 0*CMV ' fit <- sem(mod, d, ordered = names(d), std.lv = TRUE, estimator = "WLSMV") y <- c("y1", "y2", "y3") P <- lavInspect(fit, "est")$psi lavInspect(fit, "post.check") # TRUE eigen(P, symmetric = TRUE)$values # smallest -0.108 round(P[y, y], 3) # diagonal .187 .296 .207 round(cov2cor(P[y, y]), 3) # y1 with y3 is -1.081

On lavaan 0.7-2, R 4.6.1, Windows 11 x64 this gives post.check TRUE, psi's smallest eigenvalue -0.1083525, a psi diagonal of .187, .296 and .207, and a residual correlation between y1 and y3 of -1.081. The diagonal is positive and the matrix is improper only through the off-diagonal.

Two observations that may help locate it.

The miss looks like a band rather than a threshold. Pushing further, e_sd = .45 or gamma = .65, gives eigenvalues of -0.364 and -0.762, and post.check then correctly returns FALSE. Across seeds 1 to 8 at this setting, five give FALSE, two give TRUE with a proper psi, and one, seed 1, gives TRUE with an improper one. So the gross failures are caught and the marginal ones are not, which is the awkward way round: a residual correlation of 1.08 is just as impossible as one of 3.55 and much easier to read past. It also means the example sits close to the boundary, so if seed 1 does not reproduce for you, scanning a few seeds should find the band.

It also looks specific to the off-diagonal. Across the seeds above, every case that post.check does catch comes back through "some estimated ov variances are negative", never through the covariance matrix itself, and the same is true if I fix an impossible residual covariance by hand with y1 ~~ 5*y2. In the cases that slip through, no variance is negative and the matrix is non-positive-definite only through its off-diagonal structure.

ATB,
Adriaan

[Felipe Vieira]

Hi, 

In the end, I didn't find the time to further check on this, but Yves checked it and fixed it :)

Let us know if you give it a try with the development version and still find something weird for your case. 

Best, 

Felipe.