How to perform sensitivity analysis in R Lavaan package?

[Johnathan T]

Hello everyone,

I am currently trying to analyze a data set through simple mediation models (x->m->y) and double mediation models (x->m1->m2->y), and I am trying to follow the AGReMA statement guidelines as closely as possible. However, I have had no luck implementing the literature instructions and example code I found through other resources and packages for sensitivity analysis. 

Would anyone know a simple implementation for sensitivity analysis either in Lavaan or with another package I could use in tandem? I should note that my outcome variables are continuous, and my mediator variables are a mixture of binary/continuous. However, I have implemented the inverse probability weighting (IPW) method on my binary mediators to convert their values into continuous propensity scores. In other words, I am treating the all the variables in my model as continuous.

I would greatly appreciate your expertise. Thank you so much.

Sincerely,

Johnathan T

[Shu Fai Cheung (張樹輝)]

Hi,

I am not familiar with the guideline. Sensitivity analysis is a broad term instead of one specific method (e.g., Lee et al., 2018 on sensitivity in parameter values; Pek & MacCallum, 2011 on case influence). Examining the presence of influential case is one type of sensitivity analysis. If you would like to examine case influence, Prof. Mark Lai and I developed the R package semfindr for identifying influential cases in SEM, specifically, models fitted by lavaan. You can find a demonstration here:

https://sfcheung.github.io/semfindr/articles/semfindr.html

There are also other packages for influential cases in SEM:

https://cran.r-project.org/package=influence.SEM

https://cran.r-project.org/package=faoutlier

You can try them and find the package suitable for your scenario.

Hope this helps.

-- Shu Fai

Lee, T., MacCallum, R. C., & Browne, M. W. (2018). Fungible parameter estimates in structural equation modeling. Psychological Methods, 23(1), 58–75. https://doi.org/10.1037/met0000130

Pek, J., & MacCallum, R. (2011). Sensitivity analysis in structural equation models: Cases and their influence. Multivariate Behavioral Research, 46(2), 202–228. https://doi.org/10.1080/00273171.2011.561068

[Ingo Man]

Hello,

throw in my experience so far. Recently, I came across this question in the context of power analysis in SEM. I did not had the chance to do a power analysis in advance for my study, so I was interested in the power for my beta coefficients, aka, the minimum effect size I was able to detect with a reasonable high power.

In manifest analyses, many people use G*Power to do so:

In sensitivity analyses, the critical population effect size is computed as a function of Alpha, 1 - beta, and N. Sensitivity analyses may be particularly useful for evaluating published research. They provide answers to questions such as “What effect size was a study able to detect with a power of 1 - beta = .80 given its sample size and Alpha as specified by the author? In other words, what is the minimum effect size to which the test was sufficiently sensitive?” In addition, it may be useful to perform sensitivity analyses before conducting a study to see whether, given a limited N, the size of the effect that can be detected is at all realistic (or, for instance, much too large to be expected realistically).

 

Faul et al (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. https://link.springer.com/article/10.3758/BF03193146

For SEM, there is also a nice way: pwrSEM

https://github.com/yilinandrewang/pwrSEM/blob/master/README.md

I wrote with one of the authors, he answered:

But finding the minimal effect size that would have adequate power for the N you used is not dependent on your data, so it’s not post-hoc. You can use pwrSEM for this, but you’ll have to use trial-and-error. That is, input your N and alpha, pick some effect size and see what power you get, and the iterate this procedure, adjusting the effect size, until you get the power level you want.

So, the shiny-app is great, use your output to feed it and adjust your coefficients.

Found some examples:

Examples of a sensitivity power analysis

From https://doi.org/10.1016/j.intell.2021.101615

The sample size was constrained by available cases, not determined by a priori calculation. However, we established whether our sample size was sufficient for a minimum effect of interest through power analysis via simulation with 5000 iterations. Specifically, we simulated the SEM shown in Fig. 1, Fig. 2 using the covariance matrix from the Italian normative sample of the WISC-IV as the prior (Orsini et al., 2012). Concerning the minimum effects of interest, we set a coefficient for gender on the latent factors equivalent to a standardized effect of B = 0.10 (corresponding to Cohen’s d = 0.20). As power is constrained by the smallest group, and our female subsample was just over 500, we performed all simulations with N = 1000. Considering a critical α = 0.05, power was slightly suboptimal: 82% for the VCI; 73% for the PRI; 69% for the WMI; 75% for the PSI; finally, it was 80% for the g-factor. Different models were simulated for each of the effects. For a slightly larger standardized effect of interest of B = 0.15 (corresponding to Cohen’s d = 0.30), power seemed sufficient: 99% for the VCI; 98% for the PRI; 96% for the WMI; 98% for the PSI; 99% for the g-factor.

Hope that helps.

Kindly,

Marcus