Two mediation models have different results (ESEM and separate modeling)

[Serena]

Hi there,

I have two mediators in a model (i.e., IV - mediator1, mediator2 - DV).

Since the two mediators are highly correlated with each other (r = .75), I decided to use ESEM or separate modeling (i.e., IV - mediator1 - DV; IV - mediator2 - DV).

ESEM model:
I used rotation = "geomin", rotation.args = list(geomin.epsilon = 0.50).

The two mediators' correlation is .48, which is good.

Results: IV is negatively correlated with two mediators;

               mediator1 is negatively correlated with DV; and

               mediator 2 is positively correlated with DV.

Separate modeling:
Results: IV is negatively correlated with mediator1; and

                mediator 1 is negatively correlated with DV;

                IV is negatively correlated with mediator2, and 

                mediator 2 is negatively correlated with DV.

The relationship between mediator2 and DV is different in these two models.

Does anyone come across this situation?
Any thoughts about this?

Thanks,

Serena

[Christian Arnold]

Hi,

I may have misunderstood the problem. But here's a guess: One explanation could be that your mediators affect the DV with the same sign, but the partial correlation between the mediators has a different sign. In this case, it may well be that one sign is different in separate models. Suppressor models (if this is the case here) are indeed sometimes difficult to interpret meaningfully.

Why is the correlation between the mediators a problem from your point of view? As long as both variables discriminate each other, the analysis should be feasible? The IV should explain the correlation between the mediators and if this is not the case (correlated errors), then this could indicate an omitted IV.

[Serena]

Hi,

Thank you for your thoughts.

I hope these figures show the problem more clearly.

                                  Figure 1.                                                                 Figure 2.

>  Here's a guess: One explanation could be that your mediators affect the DV with the same sign, but the partial correlation between the mediators has a different sign. In this case, it may well be that one sign is different in separate models. 

    Yes, I believe these two mediators affect the DV with the same sign, as I did two separate models, the two signs (negative) are the same. I also did two mediators as two IVs (removed the original IV) and regressed on DV, the signs (negative) are the same. Importantly, the correlation between mediator 2 and DV is negative!

                     Mediator 1      Mediator 2        DV

Mediator 1            1                

Mediator 2           .75***                  1 

DV                        -.17***               -.12***

    But when I model IV, two mediators (mediator 1 ~~ mediator 2), and DV together, then the suppression effect shows up.

> Why is the correlation between the mediators a problem from your point of view? As long as both variables discriminate each other, the analysis should be feasible?

   My thought is the high correlation between two mediators causes multicollinearity. The r = .75 (> .70) of two mediators, although some people said higher than .80 or .90.

   Two mediators do discriminate each other theoretically and empirically (a. in the omega model - I calculated the omega of all items of these two mediators; b. the ESEM factor loadings showed they discriminate each other).

   Because of the high correlation of the two mediators based on my data, I think this causes multicollinearity. Then I did ESEM in a structural model, in this model, the correlation of these two mediators is r = .57, which is not that high and should not cause multicollinearity. But the regression sign (positive) between the mediator2 and DV is still different from the correlation sign (negative), please see Figure 2.

    It seems that the high correlation between these two mediators does not cause the problem. But why the signs are different? I mean the correlation sign and the beta sign. Could I still rely on the whole model? Or rely on the separate models?

> The IV should explain the correlation between the mediators and if this is not the case (correlated errors), then this could indicate an omitted IV.

   Could you explain a bit more about this?

Best,

Serena

[Christian Arnold]

Nothing special is happening here. The correlations between the mediators and the DV are negative but not very large. The correlation between the mediators is positive (and large). Suppression is taking place. The estimates move upwards if a mediator is added to the model and in your case one of the estimates exceeds zero, leading to a sign change. This will also happen with a simple multiple regression (quick and dirty based on your correlation matrix):

library(lavaan)

lower <- "
1.00
0.74  1.00
-0.17 -0.12 1.00"

cor.mtrx <- getCov(lower, names = c("m1", "m2", "y"))

parameterEstimates(sem("y ~ m1 + m2", sample.cov = cor.mtrx, sample.nobs = 100))
parameterEstimates(sem("y ~ m1", sample.cov = cor.mtrx, sample.nobs = 100))         # Not necessary, the estimate will be -0.17
parameterEstimates(sem("y ~ m2", sample.cov = cor.mtrx, sample.nobs = 100))         # Not necessary, the estimate will be -0.12

I noticed that the effects of the IV -> mediators are different in the separate models and in the model with both mediators. To me, this indicates that the model is somehow misspecified (for example, because m1 ~~ 0 * m2 is used or for some other reason).

HTH

[Serena]

Hi,

The effects of the IV -> mediators are different in the separate models and in the model with both mediators because in the model with both mediators, I used ESEM for two mediators.
If I use CFAs for mediators in the SEM, then the effects of IV -> mediators are the same as the separate models.

Yes, it is clear suppression is taking place. In this case, may I hear your thoughts on which way you will go? Is the model with both mediators or separate ones? OR any other ways that could help me to decide which one I should go? Because either way is interpretable and supported by other scholars. However, only two groups are talking about this and they have different opinions, thus I aim to do appropriate data analysis to test the relationship to get appropriate results.

Looking forward to hearing your thoughts.

Best,

Serena

[Christian Arnold]

That is difficult to answer. You can find a lot of literature on this (for example with keywords like “inconsistent models”).  Here you can find a few examples that show that such a model can sometimes be interpreted sensibly: https://www.researchgate.net/publication/228079476_An_Introduction_to_Statistical_Mediation_Analysis
Ultimately, theory should dictate which model should be used. I would intuitively (without knowing the theory in the background) use the model with both mediators and check whether there are other things going on in the background (mediation is based on assumptions, e.g. no omitted confounders).

[Serena]

Hi,

Thank you for your reply.

My understanding of the different signs of the correlation result and the regression result is that the spurious part is a lot larger than the actual causal effect part. 

Although, I tend to believe that the regression results are reliable, as it accounts for the spurious and causal relationships (we need to explain it clearly and carefully),  I am still a bit confused if the spurious part is larger, then why could we rely on the regression model? It would be great if you could add your thoughts, otherwise appreciate your time.

Best,
Serena

[Christian Arnold]

I don't know exactly what you mean by spurious. Sorrry,.

I I can't help you here.

---

Von: lav...@googlegroups.com <lav...@googlegroups.com> im Auftrag von Serena <wsere...@gmail.com>
Gesendet: Montag, Juni 3, 2024 11:19:58 AM
An: lavaan <lav...@googlegroups.com>
Betreff: Re: Two mediation models have different results (ESEM and separate modeling)

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/756f9f99-ac44-4d18-82b8-d3d4f6e65ae5n%40googlegroups.com.

[Serena]

No worries, thank you for your time!

Best,

Serena