Terrence D. Jorgensen (he, him, his)
Assistant Professor, Methods and Statistics
Research Institute for Child Development and Education, the University of Amsterdam
http://www.uva.nl/profile/t.d.jorgensen
Hi Mauricio,
I am exploring the influence of priors. Some follow-up questions for weakly informative priors:
1. Specific priors for RI-CLPMs: do you have any recommendations on the values?
e.g., NcNeish (2019; https://doi.org/10.1080/10705511.2019.1578657 suggested:
"Two-Level Dynamic Structural Equation Models with Small Samples
assuming stationarity, the magnitude of the autoregressive inertia paths from the outcome at time t – 1 to the outcome at time t must fall between −1 and 1"
"Inertia Slope Variance.—The inertia slope of the outcome (y at time t) autoregressed on the outcome at the immediately preceding time (y at time t – 1) cannot exceed 1, provided that the assumption of stationarity has been met."
2. How to set these specific priors in blavaan?
E.g., dp=dpriors(XX) , to specify Inertia Slope Variance (0,1) and autoregressive effect (-1, 1), what should XX be replaced?
3. How to do the posterior predictive checking?
I tried the following code, but it took forever and still did not give any output...
AFIs <- ppmc(fit.RICLPMEnjEff, thin = 10, fit.measures = c("srmr","chisq","rmsea","cfi"))
summary(AFIs)
Can we do something in blavaan that is similar to posterior predictive checking on the following website?
Best,
Yuan
Yuanyuan,
A couple additions to Mauricio's response:
I imagine that you have multiple slopes in your model, and the dpriors() thing sets a prior distribution for all slopes. So you might want to use prior() inside the model specification syntax. See:
https://ecmerkle.github.io/blavaan/articles/prior.html#individual-parameters
About ppmc() taking a long time: this partly depends on your model. If you have many columns in your data and/or missing data, it will take longer. I believe that future::plan("multicore") can help here (or "multisession" on windows), but it might still take awhile.
Ed
On Wed, 2023-08-16 at 07:44 -0700, Yuanyuan Hu wrote:
Hi Mauricio,
>
I am exploring the influence of priors. Some follow-up questions for
> weakly informative priors:
> 1. Specific priors for RI-CLPMs: do you have any recommendations on
> the values?
> e.g., NcNeish
> "Two-Level Dynamic Structural Equation Models with Small Samples
> assuming stationarity, the magnitude of the autoregressive inertia
> paths from the outcome at time t – 1 to the outcome at time t must
> fall between −1 and 1"
> "Inertia Slope Variance.—The inertia slope of the outcome (y at time
> t) autoregressed on the outcome at the immediately preceding time (y
> at time t – 1) cannot exceed 1, provided that the assumption of
> stationarity has been met."
> *
> *
> 2. How to set these specific priors in blavaan?
> E.g., dp=dpriors(XX) , to specify Inertia Slope Variance (0,1)
> and autoregressive effect (-1, 1), what should XX be replaced?
>
3. How to do the posterior predictive checking?
> I tried the following code, but it took forever and still did not
> give any output...
> AFIs <- ppmc(fit.RICLPMEnjEff, thin = 10, fit.measures =
> c("srmr","chisq","rmsea","cfi"))
> summary(AFIs)
> Can we do something in blavaan that is similar to posterior
> predictive checking on the following website?
> 在2023年4月3日星期一 UTC+2 14:25:55<[mauga...@gmail.com](mailto:mauga...@gmail.com)> 写道:
> > > Yuan
> > > >
> > With MCMC the results will change a little bit between runs, due to
> > MCMC error variability but should not change a lot.
> > > >
> > The main difference is due to not be running the same model, as you
> > have different number of parameters.
> > > >
> > First, check between models and find what are the differences
> > between them. Probably the fixed.x argument, you can give the same
> > argument to blavaan and see if you get the same model. Can run it
> > with very few iterations, just to see you have the right model.
> > > >
> > Once you have the same model, I would recommend BSEM with weakly
> > informative priors, these are good general applications for small
> > sample sizes. Can get properly estimated models, that dont rely on
> > the large sample assumption
> > > >
> >
> >
> > On Monday, April 3, 2023 at 11:17:11 AM UTC+2 [huyy...@gmail.com](mailto:huyy...@gmail.com)
> > > > > > > to find out why this is a problem and how to eliminate them.
> > > > > > > 2: There were 7 transitions after warmup that exceeded the
> > > maximum treedepth. Increase max_treedepth above 10. See
> > > > > > > 3: Examine the pairs() plot to diagnose sampling problems
> > > > > > >
> > > > > > > 4: The largest R-hat is NA, indicating chains have not mixed.
> > > > > > >
> > > > Running the chains for more iterations may help. See
> > > > > > > 5: Bulk Effective Samples Size (ESS) is too low, indicating
> > > posterior means and medians may be unreliable.
> > > > > > >
> > > > Running the chains for more iterations may help. See
> > > > > > > 6: Tail Effective Samples Size (ESS) is too low, indicating
> > > posterior variances and tail quantiles may be unreliable.
> > > > > > >
> > > > Running the chains for more iterations may help. See
> > > > > > > 7: blavaan WARNING: As specified, the psi covariance matrix is
> > > neither diagonal nor unrestricted, so the actual prior might
> > > differ from the stated prior. See
> > > > > > >
> > > > I attached my code and data file if you would like to try it.
> > > > > > >
> > > > Many thanks!
> > > > > > > Yuan
> > > > > > >
> > > > 在2023年4月2日星期日 UTC+2 23:57:14<[mauga...@gmail.com](mailto:mauga...@gmail.com)> 写道:
> > > > > > > > > Yuan
> > > > > > > > > >
> > > > > > In general should be colse to teach other of the model is the
> > > > same. I would recommend to use the function sem and bsem to
> > > > make sure that the same underlying constraints are used. For
> > > > example, I am not sure you are running the same model with the
> > > > argument fixed.x=T . Can check with fitMeasures of the blavaan
> > > > object to have the same npar as the lavaan object
> > > > > > > > > >
> > > > > > Also, these fit indices should be compare to the "normal"
> > > > indices not the MLR ones
> > > > > > > > > >
> > > > > > Since you are not actually sunning a multiple group model what
> > > > matters more is the total sample
> > > > > > > > > >
> > > > > > Even with models like this, would recommended to use priors
> > > > less wide as the deafult ones
> > > > > > > > > >
> > > > > > Didnt you say in an earlier post that the CFI was higher?
> > > > > > > > > >
> > > > > > ML is not neccesarily the "correct" answer, so replicating them
> > > > is not the objective of BSEM. Differences in the parameters can
> > > > also be related to the unrealistic expectation of ML that your
> > > > 130 sample is asymototically correct (equivalent to infinetly
> > > > large)
> > > > > > > > > >
> > > > > >
> > > > > >
> > > > > > On Sunday, April 2, 2023 at 8:46:50 PM UTC+2 [huyy...@gmail.com](mailto:huyy...@gmail.com)
> > > > > > > > > > > > > 在2023年3月30日星期四 UTC+2 15:12:18<[mauga...@gmail.com](mailto:mauga...@gmail.com)> 写道:
> > > > > > > > > > > > > > > You can use the BSEM fit indices paper, we have a section
> > > > > > about the interpretation of the indices and how the cutoffs
> > > > > > cannot be generalized, and you can see the otehr references
> > > > > > in that section
> > > > > > > > > > > > > > > > This is a general problem of a bad practice has become
> > > > > > common.
> > > > > > > > > > > > > > > >
> > > > > > > > > > What is "acceptable" is such a qualitative interpretation
> > > > > > that generalized the 0.9 is silly. Sorry I dont have a
> > > > > > better solution to handle reviewers
> > > > > > > > > > > > > > > >
> > > > > > > > > > On Wednesday, March 29, 2023 at 5:33:47 PM UTC+2
> > > > > > [huyy...@gmail.com](mailto:huyy...@gmail.com) wrote:
> > > > > > > > > > > > > > > > > > Running the model with lavaan and using its modification
> > > > > > > indices is indeed a fast solution.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > I tried the possible modification indices but the
> > > > > > > gammaHat is still below .900. You recommended to report
> > > > > > > gammaHat and BCFI. What if the reviewers are very
> > > > > > > critical about the the cutoff value, do you have any
> > > > > > > references that I can cite to say in this case, the model
> > > > > > > fit is acceptable?
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > Many thanks!
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > 在2023年3月29日星期三 UTC+2 16:36:08<[mauga...@gmail.com](mailto:mauga...@gmail.com)> 写道:
> > > > > > > > > > > > > > > > > > > > > Yuan
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > 1. 10000 is an overkill, if 1500 is enough to not get
> > > > > > > > the ESS warnings you are fine. It would only be
> > > > > > > > neccesary to have sucha a large sample for bayes factor
> > > > > > > > estabilibility. But not for inferences etc
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > 2. Yes, that is a warning not a problematic error
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > 3. Sadly, that is a know problem with modification
> > > > > > > > indices, they can take a while for large models, as it
> > > > > > > > is estimating the lavaan indices 600 time for example.
> > > > > > > > If R is still running I would let it run a for a bot.
> > > > > > > > or you can increase thin so that it estimates it less
> > > > > > > > times
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > out <- ppmc(fit.RICLPMpoCogEnjEff, thin=8, discFUN =
> > > > > > > > discFUN)
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > as a last resort, you can run the model with lavaan and
> > > > > > > > use its modification indices. They should not point to
> > > > > > > > different parameters. Honetsly I am not sure what would
> > > > > > > > be recommended as added parameters in this model with
> > > > > > > > so many constraints
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Wednesday, March 29, 2023 at 3:37:59 PM UTC+2
> > > > > > > > [huyy...@gmail.com](mailto:huyy...@gmail.com) wrote:
> > > > > > > > > > > > > > > > > > > > > > > > Thank you Mauricio!
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Sample:
> > > > > > > > > > > > > > > > > > > > > > > > > 1). When sample = 500, I got below warnings so I
> > > > > > > > > increased sample =1500. I was planning to use sample
> > > > > > > > > =10000 (which takes 40 mins each run) in the final
> > > > > > > > > reporting. Perhaps no need to be so high?
> > > > > > > > > > > > > > > > > > > > > > > > > 3: Bulk Effective Samples Size (ESS) is too low,
> > > > > > > > > indicating posterior means and medians may be
> > > > > > > > > unreliable.
> > > > > > > > > > > > > > > > > > > > > > > > > Running the chains for more iterations may help. See
> > > > > > > > > > > > > > > > > > > > > > > > > [https://mc-stan.org/misc/warnings.html#bulk-ess](https://mc-stan.org/misc/warnings.html#bulk-ess)
> > > > > > > > > > > > > > > > > > > > > > > > > 4: Tail Effective Samples Size (ESS) is too low,
> > > > > > > > > indicating posterior variances and tail quantiles may
> > > > > > > > > be unreliable.
> > > > > > > > > > > > > > > > > > > > > > > > > Running the chains for more iterations may help. See
> > > > > > > > > > > > > > > > > > > > > > > > > [https://mc-stan.org/misc/warnings.html#tail-ess](https://mc-stan.org/misc/warnings.html#tail-ess)
> > > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > 2) When sample = 1500, I also got below warning. I
> > > > > > > > > checked suggestions in this group, it seems that I
> > > > > > > > > can ignore this warning?
> > > > > > > > > > > > > > > > > > > > > > > > > blavaan WARNING: As specified, the psi covariance
> > > > > > > > > matrix is neither diagonal nor unrestricted, so the
> > > > > > > > > actual prior might differ from the stated prior. See
> > > > > > > > > > > > > > > > > > > > > > > > > [https://arxiv.org/abs/2301.08667](https://arxiv.org/abs/2301.08667)
> --
> You received this message because you are subscribed to the Google
> Groups "blavaan" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to [blavaan+u...@googlegroups.com](mailto:blavaan+u...@googlegroups.com).
> To view this discussion on the web visit
> [https://groups.google.com/d/msgid/blavaan/94e0ab73-591c-4c87-ae76-62575c9deaf5n%40googlegroups.com](https://groups.google.com/d/msgid/blavaan/94e0ab73-591c-4c87-ae76-62575c9deaf5n%40googlegroups.com).
Hi Mauricio,
I attached my code here and datafile (via WeTransfer, it's too big to attach here). The mdoerator is sex (binary: 1=male, 0=female). The predictor is my intervention (multicategorical, 3 groups, 0=neutral state, 1=enjoyment, 2=frustration). The outcomes are Enj, Eff, postTest, game. The model works under lavaan, but got the errors under blavaan.
The code for moderation:
RIEnj ~ d1*sex1
RIEnj ~ m11*sex1:group1 + m12*sex1:group2
RIEff ~ d2*sex1
RIEff ~ m21*sex1:group1 + m22*sex1:group2
Best,
Yuan
Chain 1: Exception: Exception: invese_spd: LDLT factor failed (in 'model_stanmarg' at line 384)(in 'model_stanmarg' at line 1835)
--You received this message because you are subscribed to the Google Groups "blavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blavaan+u...@googlegroups.com.To view this discussion on the web visit https://groups.google.com/d/msgid/blavaan/367a601e-0446-4af8-a47b-c55ae1de5526n%40googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/blavaan/d8200c87-2530-45cd-aef0-024a5628ddacn%40googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/blavaan/43df13b6-c5af-4634-814c-1b52848f8dc9n%40googlegroups.com.