Sample size in multiple mediators vs. single-mediator models

[Sabrina Twilhaar]

I'm using lavaan to test two mediation models: one with an observed variable as the mediator and one with a latent variable (3 indicators) as the mediator. In both models X is binary and Y is continuous. I have 130 observations, used FIML and bootstrap CI. As both mediators are related (r=0.58), it may be better to test a single model with two mediators instead of separate single-mediator models. This would be a more parsimonious solution. However, I find it hard to decide whether this is also a good idea given my limited sample size that affects the power and possibly the precision of the estimates. Below the results of the different models:

Model A: observed mediator M1

- path a: beta = -0.24

- path b: beta = -0.11

- path c': beta = 0.30

- indirect effect: estimate = 0.23 (95% CI -0.11, 1.01)

- direct effect: estimate =  2.51 (95% CI 1.22, 4.14)

Model B: latent mediator M2

- path a: beta = -0.50

- path b: beta = -0.53

- path c': beta = 0.07

- indirect effect: estimate = 2.21 (95% CI 0.70, 5.25)

- direct effect: estimate = 0.55 (95% CI -2.25, 2.31)

Model C: two mediators M1, M2

- path a1: beta =  -0.50

- path b1: beta = -0.68

- path a2: beta = -0.25

- path b2: beta = 0.25

- indirect effect M1: estimate = 2.85 (95% CI 0.87, 9.73)

- indirect effect M2: estimate = -0.52 (95% CI -3.37, 0.08)

- total indirect effect: 2.33 (95% CI 0.73, 7.30)

- direct effect: 0.43 (95% CI -3.93, 2.37)

How can I decide whether model C suffers from the limited sample size? I see that the CIs for the indirect effects are wider, so the estimates are less precise. What else should I look at? I cannot find simulation studies in which the "cost" in power or the sample size requirements of adding an extra mediator to the model are explored. 

Is there anyone who could give me an advise on this issue? Thanks a lot!

[Terrence Jorgensen]

I don't see what sample size has to do with this question.  The 3 models represent different theories about the causal process.  You should fit the model that allows you to test the hypotheses you have, and interpret the model that fits the data best.

Terrence D. Jorgensen

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

http://www.uva.nl/profile/t.d.jorgensen

[Sabrina Twilhaar]

Thank you for your response. Based on theory and my hypothesis, I wanted to test a model(s) with two mediators, but I thought that my sample size was a limiting factor to test larger sized models. But what I understand from your response that's a misconception? 

Op woensdag 10 april 2019 14:37:21 UTC+2 schreef Terrence Jorgensen:

[Terrence Jorgensen]

I thought that my sample size was a limiting factor to test larger sized models. But what I understand from your response that's a misconception? 

It is true that larger N leads to greater power, and the other side of the low-power coin means inflated Type II errors.  But my point is just that compared to your Model B, Model C only has 1 additional variable, so if N=130 is too few for adequate power in Model C, it is probably inadequate for Model B as well.  

Rather than relying on intuition to mislead you into fitting models that are not what you want to fit, you should address the power issue as the empirical question that it is (i.e., conduct a power analysis to see how much power you have to detect certain indirect-effect sizes when N=130).  Here is a shiny app that can facilitate (install the devtools and shiny packages first).  

http://marlab.org/power_mediation/

Although it currently only handles 2 observed mediators, I expect that any loss of power by having 3 indicators for the latent mediator (which requires estimating more parameters, but uses more data to do so) would be fairly compensated by disattenuating the estimate (i.e., removing measurement error should result in a larger estimated effect).  That is just my intuition ;-) but the point is just that the power analysis will give you an idea as to whether your power is really inadequate.  FYI, you could just run a post-hoc power analysis by plugging in correlations among the 4 variables from Model C, which you can get from this output:

lavInspect(modelC, "cor.all")

[Sabrina Twilhaar]

Thanks a lot for the elaborate response and suggestions! This is very helpful.

Op vrijdag 12 april 2019 10:16:02 UTC+2 schreef Terrence Jorgensen: