How to interpret the pValue(s) of the R SIMSEM package

51 views
Skip to first unread message

Yong Li

unread,
Nov 15, 2019, 2:23:38 AM11/15/19
to lavaan

Dear Dr Pornprasertmanit and all,

I am one of newbees of the simsem package.

When I use simsem to do a simulation like the below, I am bit confused by pValue() function.

I know p is the proportion of the number of replications that provide poorer fit (e.g., less CFI value or greater RMSEA value) than the analysis result from the observed data.

Does it mean that the data-fitted model is reliable if p are very small such as <0.05?

Is the smaller the P value the better data-fitted model?

OR just inverse?

Should I need to look at parameter coverages and powers as well?

Yong


library(simsem)
library(lavaan)

HS.model <- ' visual  =~ x1 + x2 + x3
              textual =~ x4 + x5 + x6
speed =~ x7 + x8 + x9 '

fit <- cfa(HS.model, data=HolzingerSwineford1939)

output.nomis <- sim(1000, n=nrow(HolzingerSwineford1939), model = fit, generate = fit)
plotCutoff(output.nomis, 0.05)
pValue(fit, output.nomis)

Terrence Jorgensen

unread,
Nov 15, 2019, 3:09:14 PM11/15/19
to lavaan

Does it mean that the data-fitted model is reliable if p are very small such as <0.05?

Is the smaller the P value the better data-fitted model?

OR just inverse?


As in any other null-hypothesis significance testing context, smaller p values than your alpha level imply you should reject the null hypothesis.  The null hypothesis is that your data are consistent with your model.  If very few replications produce worse fit than your observed data, then the degree to which your model fails to reproduce your data is uncommonly poor.

Terrence D. Jorgensen
Assistant Professor, Methods and Statistics
Research Institute for Child Development and Education, the University of Amsterdam

Yong Li

unread,
Nov 15, 2019, 8:20:44 PM11/15/19
to lavaan
Many thanks, Dr Jorgensen. The last sentence is hard for me to swallow.
Regards,
Yong

在 2019年11月15日星期五 UTC+11下午6:23:38,Yong Li写道:
Reply all
Reply to author
Forward
0 new messages