Test statistics in mediated relationships in lavaan

[Peter MKUMBO]

Hi there,

I am an applied researcher, and I have a small question about test statistics especially b, standard errors & z-values that lavaan produces if a researcher is interested in testing mediation. My understanding is that, the product of two paths in a simple mediated relationship does not follow a normal distribution (according to recent publications of Andrew F. Hayes) and that the only way to tell if a mediated relationship is significant or not is by using confidence intervals. However, I am wondering lavaan does produce these statistics and I don't know if they are reliable or not (I guess they are reliable but I am a little confused). A clarification would be highly appreciated.

Thanks,

Pita

[Mauricio Garnier-Villarreal]

Pita

You are correct, it cannot be assumed that the a*b term is normally distributed, but this is not new, there is a lot of literature about it.

Now, in lavaan if you do the analysis with the default features, this would lead to what is called the sobel test. Which means that the inference is made with the wald test assuming the product term is normally distributed.

There are mainly 2 ways that are recommended to deal with this, (a) use bootstrap Confidence Intervals and standard errors as bootstrap does not assume the parameter is normally distributed, for example

mod_med <- '
shs ~ cp*ple + b*grat
grat ~ a*ple

## indirect
ind := a*b
tot := a*b + cp ## total
'

fit_med_boot <- sem(mod_med, data=dat, meanstructure=T,fixed.x=F,
                    se="bootstrap", bootstrap=5000, verbose=T)
summary(fit_med_boot, standardize=T, rsquare=T)
parameterestimates(fit_med_boot, boot.ci.type="bca.simple",
                   level=0.95, standardize=T,
                   zstat = F, pvalue = F)

the second option (b) is to use the montecarlo simulation approach which is available with a function from semTools, like this

exp_ind <- 'a*b'
exp_tot <- 'a*b + cp'

mont_ind <- monteCarloMed(exp_ind, object = fit_med_sobel,
                          plot=T, CI=95)
mont_ind

mont_tot <- monteCarloMed(exp_tot, object = fit_med_sobel,
                          plot=T, CI=95)
mont_tot

[Peter MKUMBO]

Thank you so much Mauricio,

It makes a lot of sense. I was initially aware of bootstrap but I somehow thought lavaan does have auto bootstrap features when an indirect relationship is being examined. Sobel test has suffered and probably succumbed to a lot of criticisms. 

I am very appreciative of your time and help.

Pita 

[Terrence Jorgensen]

 

Sobel test has suffered and probably succumbed to a lot of criticisms. 

FYI, lavaan does not provide the Sobel test.  The Sobel test assumes that a and b have uncorrelated sampling distributions.  lavaan uses the delta method, which takes that correlation into account.  So it will work better than the Sobel test regardless, and the delta method yields nominal Type I error rates in sufficiently large samples.  The Monte Carlo method Mauricio mentioned is much less computationally intensive and works about as well as bootstrapping in smaller samples (but both break in small samples, for different reasons).

Terrence D. Jorgensen

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

http://www.uva.nl/profile/t.d.jorgensen

[Sofia Orellana]

Hi Terrence,

Just a quick question in relation to the above - would the Monte Carlo perform as well as bootstrapping in a large sample (e.g. N>10.000) ?

Thank you for your help and many useful replies in this group. 

Best, 

Sofia O.

[Terrence Jorgensen]

would the Monte Carlo perform as well as bootstrapping in a large sample (e.g. N>10.000) ?

I'm sure, but it depends on the model size.  In simple mediation models, the methods (including delta method) practically converge in samples barely over 100.

[Harmanjit Singh]

Dear Terrence,

Query 1: If there is no missing data, but data is non=normal, and we are using robust estimators like MLM in sem() to contain non-normalcy of data; then should we prefer montecarlo method to bootstrap method (since bootstrap does not run with MLR or MLM estimators)?
Query 2: 

I read following comment by you  "If both the indirect and direct effects are significant, that is consistent with partial mediation.  If only the indirect is significant, that is consistent with full mediation.  If only the direct is significant, that is consistent with no mediation." on following thread: https://groups.google.com/g/lavaan/c/1twNwxvrg7o/m/BkeAxhR8CAAJ

How to interpret full, partial or no mediation when we are using monte carlo method?
Do we have to run following codes (for the sake of examples)?:
mont_indirect <- monteCarloMed('a*b', object = fit, plot=T, CI=95)

mont_indirect
mont_direct <- monteCarloMed('c', object = fit, plot=T, CI=95)
mont_direct

mont_total <- monteCarloMed('a*b+c', object = fit, plot=T, CI=95)
mont_total

And then we compare significance of mont_indirect and mont_direct to draw conclusions regarding full, partial, or no mediation. If yes, then what is the role of calculating mont_total, i.e. total effect?

[Pat Malone]

Hi, Harmanjit.

Q1. It's a good question, but the answer is that it doesn't matter. The adjustments in MLR are to the standard errors, not the point estimates. The bootstrapped confidence intervals do not rely on the standard errors.

Q2. The total effect is a derived quantity (direct + indirect) that is often of substantive interest (X  on Y regardless of path to get there), but is not in itself part of the estimated model.

Pat

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/46d15b81-54f7-4033-89bc-b51fd3fc3010n%40googlegroups.com.

--

Patrick S. Malone, PhD

Sr Research Statistician, FAR HARBΦR

+1 803.553.4181  |  pat@  |  farharbor.com

This message may contain confidential information; if you are not the intended recipient please notify the sender and delete the message.

[Harmanjit Singh]

Thanks Patrick. 

Q1. I asked this, if I prefer to use MonteCarlo method in the above scenario and reviewer questions why you didn't use bootstrapped, what could be a better rationale to defend this choice?

Q2. Thanks

[Pat Malone]

I misunderstood. The MCMC method does use SEs, so it does matter. But you can get the robust SEs for MCMC if you're not bootstrapping, and bootstrapping doesn't use SEs, so I'm no longer sure what the conflict is.

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/c500dcff-a357-4762-aa89-6bb43b1f66f9n%40googlegroups.com.

[Harmanjit Singh]

What I interpret from your current response is that MCMC is a better approach when using robust estimator, compared to bootstrapping method. 
Kindly acknowledge whether I have understood correctly or not?

[Pat Malone]

No.  I'm saying that using a robust estimator does not affect the decision whether to use MCMC or bootstrapping. 

Patrick S. Malone, PhD
Sr Research Statistician

FAR HARBΦR
+1 803.553.4181 | pat@ | farharbor.com
This message may contain confidential information; if you are not the intended recipient please notify the sender and delete the message.

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/4abda7f0-5ce4-477a-b322-133517aad702n%40googlegroups.com.

[car...@web.de]

Harmanjit, If you simply want to know which approach is the "better" one, then just look at what is the common one in your field. I suspect it is bootstrapping.

Am 11.06.21, 19:35 schrieb 'Pat Malone' via lavaan <lav...@googlegroups.com>:

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/CAJhmz4eZSrbkyhViMWQitX6Mnjok%3D5Z%3DMTjsMwnyTB5GFFCbVg%40mail.gmail.com.

[Harmanjit Singh]

Thanks Patrick

[Harmanjit Singh]

Yes you are right. Bootstrapping is more common in marketing domain. 

[car...@web.de]

As a supplementary remark to Pat.
Basically you can also bootstrap with MLR. For example, by using the boot package. However, this is pointless. First, the procedure takes much longer. Second, the results do not differ, since MLR corrects the s.e. and the chi square. Both are irrelevant for your problem, since only the point estimates are needed.

--
Diese Nachricht wurde von meinem Android Mobiltelefon mit WEB.DE Mail gesendet.

Am 12.06.21, 10:22 schrieb Harmanjit Singh <harman...@gmail.com>:

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/5a9ac8bc-ac2a-47d4-908c-1ac168edee70n%40googlegroups.com.

[Harmanjit Singh]

Thanks, in that case I have two choices:
1. MLM + MCMC (as you said MLM + Bootstrap can be done, but doesn't make sense)

2. ML + Bootstrap

I am keen on using MLM, so probably option 1 suits me better despite it being not a much common choice in my field, however what I am really interested in knowing is that whether going with this combination will lead to counter questions from reviewers on why did I use this or why I did not go with option 2?
Looking forward to your comments. 

[Pat Malone]

Or you can use MLM(R) for point estimates and fit and the bootstrap for confidence intervals. Typically, that's what you'd do with ML anyway. The bootstrap point estimates are only asymptotically equivalent to the computed point estimates from ML (any type, they're all the same), so with finite resamples you're better off with the ML estimates (assuming assumptions are met). 

The purpose of the bootstrap--or MCMC CIs--is the confidence limits. As I've been saying, one choice does not lock you into the other. 

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/553ee52a-1f4e-4a9f-bf8d-c5ee3e55d06an%40googlegroups.com.

[Terrence Jorgensen]

Query 1: If there is no missing data, but data is non=normal, and we are using robust estimators like MLM in sem() to contain non-normalcy of data; then should we prefer montecarlo method to bootstrap method (since bootstrap does not run with MLR or MLM estimators)?

If you requested robust SEs, then the Monte Carlo method for CIs will use the robust sampling covariance matrix, returned by the vcov() method, so yes, nonnormality will be accounted for.  In sufficiently large samples (depends how large your model is), Monte Carlo and bootstrapping should yield the same results, but Monte Carlo would not be computationally intensive, so you would get your answer in a couple seconds instead of a couple minutes.  Regardless of industry standards, you can cite the paper on the monteCarloMed() help page to support your choice.  From what I have observed, the hype about bootstrapping tends to be based on minute differences in small samples.

Note that in the development version of semTools, monteCarloMed() is replaced by monteCarloCI() because it is a more general method than simply for mediation models.  Also, you also have the option of simply passing your fitted lavaan model with user-defined parameters, so the function will automatically detect those and return CIs for all user-defined parameters.

devtools::install_github("simsem/semTools/semTools")

[Harmanjit Singh]

Thanks Patrick

[Harmanjit Singh]

Thanks Terrence for your suggestions.
I installed the development version of semTools (version 0.5-4.915), however, on its help page, none of the links are workable. Upon clicking any link of function, it is showing "Internal Server Error". So I am unable to see examples related to usage of monteCarloCI () 
Kindly share its usage example here.

[Terrence Jorgensen]

Upon clicking any link of function, it is showing "Internal Server Error"

Did you install semTools in a fresh R session, so it did not conflict with a previously installed version?  That could be a problem if the old version is currently loaded or even just in the PATH.  You can check sessionInfo() to verify semTools is not currently loaded before installing.  Sometimes it is also necessary to shut R down and restart it before loading the development version of the semTools library.  

[Harmanjit Singh]

Yes, that was the problem.
Sorted now. Thanks!