Multilevel path analysis - Lavaan - Residuals T1 and T2

[Ayşenur]

Dear all, 

I'm conducting a multilevel path analysis by using Lavaan and I have the outcome variables as residuals of T1 (sqa_interest) and T2 (sqb_pint) variables (e.g., interest). I've calculated the residuals at student level (level 1) and standardized it by using the code here:  https://www.statology.org/standardized-residuals-in-r/

model1 <- lm(sqb_pint ~ sqa_interest, data=mydata, na.action = na.exclude)

standard_res_interest2 <- rstandard(model1)

Now, I'm wondering how to calculate it at classroom level (level 2) and use that variable in the syntax of two-level multilevel path models by using lavaan. (output the residuals for level 1 and level 2 separately) I would have to slightly adjust the syntax for the models, using the level-specific residuals (for level 1 the residuals of level 1 and for level 2 the residuals of level 2). An example syntax for the model is below. 

Model_direct_cog <- '
    level: 1
    standard_res_interest2 ~ sqa_cog
     level: 2
    standard_res_interest2 (?) ~ sqa_cog
             '
fitModel_direct_cog  <- sem(Model_direct_cog, data = mydata, cluster="IDTEACHER", se = "robust.huber.white")
summary(fitModel_direct_cog, fit.measures = TRUE, standardized = TRUE, rsquare = TRUE)

Thanks a lot and I appreciate your help!

Aysenur

 

[Terrence Jorgensen]

You don't need to partial out sqa_cog ahead of time, just include it as a covariate to obtain the partial regression slope of interest at each level.  That's what a partial regression slope is (even in single-level data).

Model_direct_cog <- '
   level: 1
     sqb_pint ~ sqa_cog + sqa_interest

   level: 2
     sqb_pint ~ sqa_cog + sqa_interest
'

A further advantage of doing it this way in ML-SEM is that you don't have to calculate (potentially unreliable) cluster means to separate the level-1 and level-2 effects.

https://doi.org/10.1080/10705511.2011.557329

Terrence D. Jorgensen

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

http://www.uva.nl/profile/t.d.jorgensen

[Ayşenur]

Dear Terrence,

Thank you so much for your reply. I, indeed, first tried to include T1 achievement and interest as covariates (of an independent variable), three mediators, and include T2 achievement and interest as outcomes. However, for the three partial mediation models, this approach did not work. Later, I used the same model only one mediator and they did not work either. That's why I tried to use residuals. 

The longitudinal mediation models had low fit indices and I received the following warning: 

Warning message:

In lav_model_vcov(lavmodel = lavmodel, lavsamplestats = lavsamplestats,  :
  lavaan WARNING:
    The variance-covariance matrix of the estimated parameters (vcov)
    does not appear to be positive definite! The smallest eigenvalue
    (= -6.950311e-13) is smaller than zero. This may be a symptom that
    the model is not identified. 

One example syntax of those mediation models were as follows: 

Model_indirect_cog_long <-
  '
    level: 1
              sqa_usecogact  ~ a1*sqa_cogdisc
              sqa_usetot     ~ b1*sqa_cogdisc
              sqa_useselfdet ~ c1*sqa_cogdisc
             
              sqb_ach  ~  d1*sqa_usecogact + e1*sqa_usetot + f1*sqa_useselfdet
              sqb_pint  ~  g1*sqa_usecogact + h1*sqa_usetot + j1*sqa_useselfdet
             
              sqb_ach  ~ k1*sqa_cogdisc
              sqb_pint ~ l1*sqa_cogdisc
             
              sqb_ach ~ p1*sqa_ach
              sqb_pint ~ r1*sqa_interest
             
              sqa_usecogact ~~ sqa_useselfdet + sqa_usetot
              sqa_useselfdet ~~ sqa_usetot
             
              sqa_ach ~~ sqa_cogdisc
              sqa_interest ~~ sqa_cogdisc
              sqa_ach ~~ sqa_interest
             
              a1d1 := a1*d1
              a1g1 := a1*g1
             
              b1e1 := b1*e1
              b1h1 := b1*h1
             
              c1f1 := c1*f1
              c1j1 := c1*j1
             
    level: 2
              sqa_usecogact  ~ a2*sqa_cogdisc
              sqa_usetot     ~ b2*sqa_cogdisc
              sqa_useselfdet ~ c2*sqa_cogdisc
             
              sqb_ach  ~  d2*sqa_usecogact + e2*sqa_usetot + f2*sqa_useselfdet
              sqb_pint  ~  g2*sqa_usecogact + h2*sqa_usetot + j2*sqa_useselfdet
             
              sqb_ach  ~ k2*sqa_cogdisc
              sqb_pint ~ l2*sqa_cogdisc
             
              sqb_ach ~ p2*sqa_ach
              sqb_pint ~ r2*sqa_interest
             
              sqa_usecogact ~~ sqa_useselfdet + sqa_usetot
              sqa_useselfdet ~~ sqa_usetot
             
              sqa_ach ~~ sqa_cogdisc
              sqa_interest ~~ sqa_cogdisc
              sqa_ach ~~ sqa_interest
             
              a2d2 := a2*d2
              a2g2 := a2*g2
             
              b2e2 := b2*e2
              b2h2 := b2*h2
             
              c2f2 := c2*f2
              c2j2 := c2*j2
'

fitModel_indirect_cog_long <- sem(Model_indirect_cog_long, data = mydata, cluster="IDTEACHER", se = "robust.huber.white")
summary(fitModel_indirect_cog_long, fit.measures = TRUE, standardized = TRUE, rsquare = TRUE)

Do I miss something or do you suggest any other ideas for this model? Thanks a lot in advance!

Best,

Ayşenur

Virüs yok.www.avast.com

Terrence Jorgensen <tjorge...@gmail.com>, 1 Eyl 2022 Per, 13:58 tarihinde şunu yazdı:

--
You received this message because you are subscribed to a topic in the Google Groups "lavaan" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lavaan/2ZBfdAMACbE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/3e00d69a-dd41-418f-b035-41d868e151c6n%40googlegroups.com.

[Terrence Jorgensen]

for the three partial mediation models, this approach did not work. Later, I used the same model only one mediator and they did not work either

What does that mean?  No convergence?  Starting with a smaller model before building up was a good idea.  Try going even smaller, to see if a multiple regression "works".  If not, you can try the same model in lme4::lmer() to see if it "works" there.

How many L1 and L2 units?  Range of cluster sizes?  

Why request robust se= but not test= (or simply shortcut estimator = "MLR")?  Saturated model, I suppose...

You can remove the (co)variances of exogenous predictors, so that their (level-specific) summary stats are treated as fixed, leaving less for the optimizer to do.  

[Michael Zyphur]

Hi Ayşenur

Good question. These kinds of problems are very common in multilevel path models and SEM. In addition to what Terrence has asked and said, usually the steps I recommend with Multilevel SEM is to 1) check ICCs, which you can do with lavInspect( fitModel_indirect_cog_long , "icc") and; 2) check the W and B correlations which you can do with a saturated model at both levels and requesting standardized output.

For more information on these types of models and some of the problems you'll encounter and how to address them, the Instats seminar on Multilevel SEM in R covers this topic in some detail, in addition to other relevant features of these models and their interpretation.

Hope this helps!

Mike

[Ayşenur]

Dear Terrence and Mike, 

Thank you very much for your answers, questions, and the additional information. These are very helpful. I tried to answer your questions below.

L1 sample size: 958 students (Number of used observations: 799)

L2 cluster : Number of clusters: 40

Range of cluster size 11-31

Those mediation models did not initially "work", because they had low fit indices. However, in the models some paths were missing, not saturated (T1 interest to all three mediators, T1 achievement to all three mediators). Then, I came up with the idea of including the paths from those covariates to the mediators. After that, the fits of the three models improved.   

However, I was wondering why I still receive the following warning: 

Warning message:

In lav_model_vcov(lavmodel = lavmodel, lavsamplestats = lavsamplestats,  :

  lavaan WARNING:

    The variance-covariance matrix of the estimated parameters (vcov)

    does not appear to be positive definite! The smallest eigenvalue

    (= -4.462518e-13) is smaller than zero. This may be a symptom that

    the model is not identified.

Is it because of the convergence problem even that the mediation models fit? I don't receive any warnings for the direct effect models (with less variables). Or is it a machine precision problem? 

I also tried the models with MLR instead of se = "robust.huber.white" with ML. The model fits were slightly better with MLR. In this case, is it better to use MLR even that the models are not saturated?  

I had calculated within/between level correlations and the ICC's:  

Between Correlations : Range -.02 and .59

Within Correlations: Range -.03 and .62

I've calculated the ICC's with ICC1.lme() function;

ICC's of IV's: .10 .37 .26 

ICC's of Mediators : .04 .04 .11

ICC's of covariates : .13 .20

ICC's of outcomes: .12 .16

However when I calculate the ICC's with lavInspect(fitModel, "icc") for three mediation models, some values are slightly different from above. 

For example, 

ICC's of IV's : # sqa_cogdisc 0.105 sqa_classman 0.378   sqa_support 0.264

ICC's of Mediators : # sqa_usecogact 0.056  sqa_usetot 0.023  sqa_useselfdet 0.114

ICC's of covariates: # sqa_interest  0.135 sqa_ach 0.197    

ICC's of outcomes: # sqb_pint  0.106 sqb_ach  0.138

How can I refer to the values from the lavInspect (from which models)?

Thank you for your help!

All the best,

Ayşenur

Ayşenur <alp.a...@gmail.com>, 12 Eyl 2022 Pzt, 18:37 tarihinde şunu yazdı: