modification indices suggest converting covariance to regression and vis versa?

[matta...@gmail.com]

After fitting a model, we received a surprisingly poor fit.

When running the lavaan::modificationIndices() on the model, I get the expected list of MIs, but this contains two surprising suggestions:

1. My model contains A ~ B, and the MI suggests A ~~ B.
2. My model contains X ~~ Z, and the MI suggests X ~ Z.

Neither make sense, theoretically, but I have never seen anything like this before so I was wondering if this actually reflects something deeply wrong with my model?

Or is it possible for MIs to suggest switching between covariances and regressions and vis-versa?

Thanks,

Mattan

[Shu Fai Cheung]

Dear Mattan,

This is normal, to me. I see suggestions like this in both lavaan and AMOS, even for data I generated from a known population model and simple path analysis models.

An SEM program cannot decide what makes sense and what does not. You can read p. 284 of Kline (2016) for cautions regarding MI

Kline, R. B. (2016). Principles and practice of structural equation modeling (4th Ed.). The Guilford Press.

In my opinion, if MIs are to be used, they should only be used as, well, suggestions. The SEM programs, lavaan and others, are ignorant of the theoretical meanings of a model (maybe until we have AI-powered SEM programs). We should use MIs based on theoretical ground, and override the ordering suggested by MI if a change with smaller MI makes more theoretical sense, e.g., lead to a new plausible hypothesis that should be tested in further studies.

Hope this helps.

Regards,

Shu Fai Cheung (張樹輝)

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/bef94de9-302f-4a4c-a5b6-66fff39c147bn%40googlegroups.com.

[Edward Rigdon]

Mattan--

     I think the "problem" lies elsewhere in your model. The MIs can only test "constraints," not parameter estimates that are already free, so if X ~~ Y is free, but something else in the model is constraining covariance involving X and Y, then the program might suggest X ~ Y, or maybe Y ~ X. Lacking any specifics, you might look for a constrained connection between one variable that is "upstream" of the X, Y pair and one that is "downstream."

     But also remember that modification indices are often redundant, in the sense that multiple indexes will relate to the same actual discrepancy between model and data. So a different modification index may already be telling you exactly what modification would resolve the problem. But that is how the world works, anyway--there is a blockage of traffic on a highway, and a web of local streets are jammed with the traffic that is being diverted off the highway. The core problem is the blockage on the highway, but "traffic modification indices" would make suggestions aimed at clearing traffic on all the side streets.

--Ed Rigdon

[matta...@gmail.com]

Thanks to both of you!

I am treating MIs as suggestions only, but I was surprised to see it suggesting an "arrow" that was already somewhat present in my model.

Ed - your answer makes sense to me, there are some other constraints in the model (a longitudinal analysis), but I can't see how those constraints are related to the X~~Y covariance. Very odd - will investigate further.

(Great analogy btw)