naive bootstrap standard errors and Bollen and Stine boostrap chi-sq p-value

[stephen sefick]

Hello all:

I am conducting mediation analysis with lavaan. I am new to SEM (1.5 months), but not to R. In advance, I appreciate any insight into my questions.

Here is my question:

Naive or Bollen Stine Boostrap to make inferences on indirect/total/direct effects parameter estimates (i.e., sem(model, data, se="bootstrap") followed by bootstrapping the p-value or sem(model, data, se="bootstrap", test="bollen.stine"))

I am waiting on Bollen and Stine's 1992 paper (my library doesn't have access to this paper). I don't understand the difference/implications at the moment.

Thank you all for the help in advance. I can provide further details/R code/etc. if needed.
kindest regards,

Stephen Sefick

[Terrence Jorgensen]

The Bollen-Stine (or "model-based") bootstrap rescales the observed data so that they are consistent with the null hypothesis of perfect fit (i.e., it begins by transforming the data so that the observed means and covariance matrix are identical to the model-implied means and covariance matrix).  This is so that the bootstrapped distribution of chi-squared statistics (and by implication, the p-value/probability of a chi-sq at least as large as your observed chi-sq) is consistent with the null hypothesis.  You do not use Bollen-Stine method to calculate SEs or CIs for your parameters.

If you can't obtain the Bollen-Stine article because your library doesn't have access to Sociological Method and Research, perhaps they have access to Multivariate Behavioral Research.  A 2009 article (URL below) reviews this method and generalizes it for use with missing data.

dx.doi.org/10.1080/00273170903333590

Terry

[stephen sefick]

Terry,

Thank you so much for the very helpful response! I will try to get the paper you mention (URL) to read about the method further.  I hope that you have a wonderful Saturday!

many thanks,

Stephen

--
You received this message because you are subscribed to a topic in the Google Groups "lavaan" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lavaan/oq268s_4FhM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lavaan+un...@googlegroups.com.
To post to this group, send email to lav...@googlegroups.com.
Visit this group at http://groups.google.com/group/lavaan.
For more options, visit https://groups.google.com/d/optout.

--

Stephen Sefick
**************************************************
Auburn University                                        
Biological Sciences                                      
331 Funchess Hall                                       
Auburn, Alabama                                        
36849                                                          
**************************************************
sas...@auburn.edu                                  
http://www.auburn.edu/~sas0025                
**************************************************

Let's not spend our time and resources thinking about things that are so little or so large that all they really do for us is puff us up and make us feel like gods.  We are mammals, and have not exhausted the annoying little problems of being mammals.

                                -K. Mullis

"A big computer, a complex algorithm and a long time does not equal science."

                              -Robert Gentleman

[yrosseel]

On 05/31/2014 10:15 PM, Terrence Jorgensen wrote:
> The Bollen-Stine (or "model-based") bootstrap rescales the observed data
> so that they are consistent with the null hypothesis of perfect fit
> (i.e., it begins by transforming the data so that the observed means and
> covariance matrix are identical to the model-implied means and
> covariance matrix). This is so that the bootstrapped distribution of
> chi-squared statistics (and by implication, the p-value/probability of a
> chi-sq at least as large as your observed chi-sq) is consistent with the
> null hypothesis. You do not use Bollen-Stine method to calculate SEs or
> CIs for your parameters.

Note that if you specify se="bootstrap", and test="bollen-stine", then
'bollen-stine' is used for both SEs and the test statistic. But for the
SEs, this has (almost) no impact.

Yves.

[Sam]

Hi there, 

I was running the boostrapLavaan example and I found that the bootstrap chisq results were quite different from the original chisq results. My understanding from above is that the Bollen Stein bootstrap transform the data matrix so that simulation of the goodness-of-fit indices will be similar.  

I tried again on a second dataset and that seemed fine. So I was wondering  1.) if I was the only one who got large differences in the chisq for the example or 2.) whether I am missing an important piece of information that led to the large differences?? 

I would appreciate some advice. 

Thank you

> T.orig <- fitMeasures(fit, "chisq")

> T.orig

 chisq 

85.306 

> T.boot <- bootstrapLavaan(fit, R=10, type="bollen.stine", FUN=fitMeasures)

> T.boot$chisq

 [1] 23.67527 24.89352 14.00134 19.80881 23.65160 25.55365 31.60391 21.72346 32.15879 23.76146

> mean(T.boot$chisq)

[1] 24.08318

==========================

(Second dataset)

> fitMeasures(m.model.fit, "chisq")

 chisq 

66.431

run.1 <- bootstrapLavaan(m.model.fit, R = 10,FUN=fitMeasures, type="bollen.stine")

> run.1$chisq

 [1] 62.74551 52.14190 46.75373 80.39481 61.30703 57.09985 58.42835 49.65539 50.42925 63.15232

> mean(run.1$chisq)

[1] 58.21081

[Terrence Jorgensen]

My understanding from above is that the Bollen Stein bootstrap transform the data matrix so that simulation of the goodness-of-fit indices will be similar.  

The data are transformed to be consistent with the null hypothesis--that is, so that the observed covariance matrix and mean vector are equal to their model-implied analogs.  Then the data are bootstrapped to empirically approximate the sampling distribution of the chi-squared statistic under the null hypothesis, and use it to calculate an approximate p value.  The bootstrap distribution (and the mean chi-squared) will therefore be different if your model-implied moments are substantially different from your observed sample moments.

I tried again on a second dataset and that seemed fine.

When you use only R=10 bootstrap sample, there will be a LOT of sampling variability from one iteration to the next.  That's why you want to set it to ~2000 (if that's feasible for your computer).

Terry