How to interpret the pValue(s) of the R SIMSEM package

[Yong Li]

Dear Dr Pornprasertmanit and all,

I am one of newbees of the simsem package.

When I use simsem to do a simulation like the below, I am bit confused by pValue() function.

I know p is the proportion of the number of replications that provide poorer fit (e.g., less CFI value or greater RMSEA value) than the analysis result from the observed data.

Does it mean that the data-fitted model is reliable if p are very small such as <0.05?

Is the smaller the P value the better data-fitted model?

OR just inverse?

Should I need to look at parameter coverages and powers as well?

Yong

library(simsem)
	library(lavaan)
	HS.model <- ' visual  =~ x1 + x2 + x3
	textual =~ x4 + x5 + x6
	speed =~ x7 + x8 + x9 '
	fit <- cfa(HS.model, data=HolzingerSwineford1939)
	output.nomis <- sim(1000, n=nrow(HolzingerSwineford1939), model = fit, generate = fit)
	plotCutoff(output.nomis, 0.05)
	pValue(fit, output.nomis)

[Terrence Jorgensen]

Does it mean that the data-fitted model is reliable if p are very small such as <0.05?

Is the smaller the P value the better data-fitted model?

OR just inverse?

As in any other null-hypothesis significance testing context, smaller p values than your alpha level imply you should reject the null hypothesis.  The null hypothesis is that your data are consistent with your model.  If very few replications produce worse fit than your observed data, then the degree to which your model fails to reproduce your data is uncommonly poor.

Terrence D. Jorgensen

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

http://www.uva.nl/profile/t.d.jorgensen

[Yong Li]

Many thanks, Dr Jorgensen. The last sentence is hard for me to swallow.

Regards,

Yong

在 2019年11月15日星期五 UTC+11下午6:23:38，Yong Li写道：