Running a power analysis using lavaan/semTools

[Aron Lindberg]

Hi All,

I am trying to run a power analysis for a lavaan model, but I'm not quite sure how to do it. I've posted my question on StackOverflow:

https://stackoverflow.com/questions/60996568/running-a-power-analysis-on-a-lavaan-latent-growth-curve-model

I'm repeating it below for your convenience. Any help would be deeply appreciated!

I am trying to conduct a power analysis using `semTools` on a latent growth curve model estimated using `lavaan`. See below:

    library(RCurl)

    library(lavaan)

    library(semTools)

    x <- getURL("https://gist.githubusercontent.com/aronlindberg/dfa0115f1d80b84ebd48b3ed52f9c5ac/raw/3abf0f280a948d6273a61a75415796cc103f20e7/growth_data.csv")

    growth_data <- read.csv(text = x)

    model_regressions <- ' i =~ 1*t1 + 1*t2 + 1*t3 + 1*t4 + 1*t5 + 1*t6 + 1*t7 + 1*t8 + 1*t9 + 1*t10 + 1*t11 + 1*t12 + 1*t13+ 1*t14 + 1*t15 + 1*t16 + 1*t17 + 1*t18 + 1*t19 + 1*t20

    s =~ 0*t1 + 1*t2 + 2*t3 + 3*t4 + 4*t5 + 5*t6 + 6*t7 + 7*t8 + 8*t9 + 9*t10 + 10*t11 + 11*t12 + 12*t13 + 13*t14 + 14*t15 + 15*t16 + 16*t17 + 17*t18 + 18*t19 + 19*t20

    # fixing error-variances

    t8 ~~ 0.01*t8

    t17 ~~ 0.01*t17

    t18 ~~ 0.01*t18

    # regressions

    s ~ h_index

    i ~ h_index'

    SSpower(powerModel = model_regressions, popModel = model_regressions, n = c(87, 125), fun = "growth")

This, however, does not seem to work. My overall question is: how do I run a power analysis using `semTools` for a latent growth curve model estimated using `lavaan`? And more specifically, what should I use to specify `powerModel` and `popModel`?

Best,

Aron

[Nickname]

Aron,
  Caveat:  My answer is based entirely on the SSpower() help file.

  The first question is what you wish to obtain the power for.  SSpower() is designed to evaluate the statistical power to reject the null hypothesis that  given parameter (or set of parameters) is equal to a specific value (e.g., zero) in the population.  If that is not what you want, you may  have the wrong function.  If it is what you want, then you need to decide which parameter or parameters you wish to evaluate the power for.

  The key ingredients in any power analysis are the value of the parameter being estimated (a.k.a. effect size), the alpha level, and the sample size.  The latter two of these are specified in the SSpower() call using the n and alpha parameters.

  The purpose of the population model is to specify the population parameter values.  Just as if you were using simulateData(), you should specify the values of all of the parameters in the model.  There should be no free parameters because the value of power could vary as a function of any parameter.  You are essentially evaluating the counterfactual: How much power would I have for a given (set of) parameters if the world worked a certain way.  Your population model is specifying the way the world works according to the antecedent (i.e., condition) of this conditional.

  The purpose of the power model (or analysis model) is to specify three things:  (a) the model you plan to fit to the data, (b) the parameter(s) for which you want to conduct the power analysis, and (c) the values posited by the null hypothesis.  You do the latter two by fixing the parameter(s) of interest to the desired values (e.g., zero for a nil null hypothesis test).  Parameters other than those being evaluated and those fixed in the intended analysis can be left free in the power model.

  If you are interested in the power of more than one parameter, that is one power calculation for a set of parameters, then specify the number of parameters using the nparam parameter to the SSpower() function.

  The result of executing the function call should be a single value representing the statistical power for the above specifications.  It may be helpful to compare the above description to the example provide in the help file before attempting your own analysis.  The SSpower function can also handle vectors of values, at least for the n parameter.

Keith
------------------------
Keith A. Markus
John Jay College of Criminal Justice, CUNY
http://jjcweb.jjay.cuny.edu/kmarkus
Frontiers of Test Validity Theory: Measurement, Causation and Meaning.
http://www.routledge.com/books/details/9781841692203/

[Aron Lindberg]

Thanks Keith,

Thanks--this brings me part of the way. I am indeed looking to estimate the statistical power to reject the null hypotheses that the two groups in the data (group = "type") have the same parameter values for slope, variance of the slope, and the coefficients for h_index on s and i. n is c(87, 125), and alpha can be set to .05. I'm assuming that powerModel should be the actual analytical model I'm testing. I still, however, have no idea how to specify popModel. Any advice?

Best,

Aron

[Nickname]

Aron,
  I am sorry that my explanation was not successful.  I tried but apparently failed to explain why it is not true that "... powerModel should be the actual analytical model I'm testing."  Using that model, one would fail to tell SSPower() which parameters to evaluate for power.  Fortunately, I do not think any of that matters.

  I tried SSPower with a multiple group analysis and it does not appear to support multiple groups.  So, my suggestion would be that you switch to using findRMSEApower() instead.  You can use 0 as your value for rmsea0 to test perfect fit.  The trick is to specify rmseaA, which is the value of RMSEA for the model you plan to fit under the conditions (the way the world works) specified in your power analysis.  The article cited in the help file may be of some assistance here.  One strategy might be to specify the population model according to the desired power analysis, calculate the implied population covariance matrix, and then fit the model from the analysis to that population matrix.  Be sure to specify group=2 when using findRMSEApower().

  It seems to me that your question about the population model conflates two issues:  the model you want to specify and the syntax needed to specify it.  These are separate questions.  If you are unsure of the first, then you do not yet have a well specified power question.  You need to fully spell out the conditions under which you want to calculate power.  For a one-sample t test, this is as simple as specifying the population mean and standard deviation.  For SEM, it requires you to specify all the parameter values in the model, just as if you were using simulateData().  Just like the power to reject Mu = 0 differs depending upon the value of Mu, so the power to reject a model differs depending upon the population model.

  Once you decide on the population model for your power analysis, you can specify the model by premultiplying each parameter by a vector of the form c(a,b) replacing a and b with the parameter values for group1 and group2.  You can find an example in the third box on the following page.

http://lavaan.ugent.be/tutorial/groups.html

Good luck,

[Nickname]

Aron,
  Two further thoughts:

1. Just to be clear, although the SSpower() power model is not the model used in the analysis, it is derived from it.  You start with that model and fix the parameters of interest.

2. If you are really stuck formulating your research question for the power analysis, you can always transform a power analysis into a sensitivity analysis by running a series of analyses for different values of some parameter.  findRMSEApower() boils down the antecedent of the power-analysis conditional to one number, the alternative RMSEA value.  Instead of reporting a power value, you could instead report a power curve for various values of the alternative RMSEA.

[Alex Schoemann]

Hi Keith and Aaron,

For SSpower you can specify a multiple group model, but you need to specify a list of covariance matrices and mean vectors for each group using the Sigma and mu arguments (instead of the popModel argument), the help file for SSpower has an example of a multiple group model (including how to get population covariance matrices and means from lavaan syntax). 

Aaron, everything Keith mentioned about determining values in the population model is correct. It's one of the most difficult parts of computing power for SEM models. If you have those values, you might also consider power analysis via Monte Carlo simulations. The simsem package provides a framework for Monte Carlo simulations based on lavaan syntax.

Alex