2-1-1 MSEM: Model Fit Indices CFI and TLI appear as NA due to no degrees of freedom left - how to resolve??

[Andreas G.]

Dear all,

I have the following 2-1-1 MSEM mediation model with Responsible Leadership as independent and Performance and Psychological Wellbeing as dependent variables:

model211 <- "

    level: 1

        Performance ~ Psychological_Wellbeing + Age + Gender_dummy + Organization_1 + Organization_2 +

        Organization_3 + Organization_4 + Organization_5 + Organization_6 + Organization_7 +

        Organization_8 + Organization_9

    level: 2

        agPerformance ~ b*agPsychological_Wellbeing + c*Responsible_Leadership

        agPsychological_Wellbeing ~ a*Responsible_Leadership

        # indirect and total effects

        ab:=a*b

        total:=ab+c

"

The model works well and all relationships are significant. However, the CFI and TLI indices do not show (appear as NA) and I think it is because I have 0 degrees of freedom. The problem surely occurs because of these 9 organisation dummies, but I need them as controls.

According to Mair(2018) those two indices are anyways of limited use in multilevel SEMs - he purports that one should only look at chi-square - however, in any case, it looks quite ominous that they show "NA" and I would like to fix this (also, and I assume for the same reason, lavaan tells me that the baseline model estimation failed) -- any suggestions on how I can solve it??

- I attached a screen shot of the entire output -

Thanks for any help in advance, much appreciated!

Andreas

[Terrence Jorgensen]

CFI and TLI indices do not show (appear as NA) and I think it is because I have 0 degrees of freedom. 

Yes, your model is saturated, so fit is arbitrarily perfect.  What is there to resolve?

Terrence D. Jorgensen

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

http://www.uva.nl/profile/t.d.jorgensen

 

[Andreas G.]

Hello Terrence,

thank you very much for your answer!

If I am not mistaken, a saturated model is not of much use statistically when my goal is to do an MSEM to test my hypotheses as the first answer in this discussion here shows.

Now, I realised that I can get a non-saturated model by adding my measurement model at each level, like so for the variable Psychological_Wellbeing:

                # wellbeing factors

                life_satisfaction =~ LS1 + LS2 + LS3 + LS4 + LS5

                meaningfulness =~ MF1 + MF2 + MF3 + MF4 + MF5 + MF6

                PANAS =~ PANASa1 + PANASa2 + PANASa4 + PANASa7 + PANASa8

                thriving =~ THRa1 + THRa2 + THRa3 + THRa4

                Psychological_Wellbeing =~ meaningfulness + PANAS + life_satisfaction + thriving

Doing this, I get over 2947 degrees of freedom, so a non-saturated model. However, Lavaan tells me it is not able to compute the baseline model, which is why I still get "NA" for CFI and TFI. Do you have an idea why that is? Maybe I can just define my own baseline model?

My best regards,

Andreas

[Terrence Jorgensen]

If I am not mistaken, a saturated model is not of much use statistically when my goal is to do an MSEM to test my hypotheses as the first answer in this discussion here shows.

Holger's answer there was a little oversimplified.  I imagine your hypotheses are about the relationships (regression slopes) among variables, in which case they are testable.  The summary() output provides Wald z statistics of the H0 that each parameter is zero, which are asymptotically equivalent to LRTs that could be calculated by actually constraining a path to zero (giving your model df=1, with a chi-squared value quite similar to the squared Wald z).  Holger's example of switching the direction of arrows is an issue of equivalent models, which is a problem that exists whether your model has df or not.  

I still get "NA" for CFI and TFI. Do you have an idea why that is?

Try it again with the latest development version, in case that is a bug that got resolved (I recall something similar, not sure if it is the same issue).

install.packages("lavaan", repos = "http://www.da.ugent.be", type = "source")

  

Maybe I can just define my own baseline model?

Sure.

[Andreas G.]

Hello Terrence,

thank you very much for the explanation, that makes perfect sense. In fact, I am very happy that I can use my saturated model after all as - for some reason - the non-saturated models where I define my first- and second-order variables are much different and do not show the same, expected results (a curious thing to me).

thanks again,

Andreas

[Terrence Jorgensen]

for some reason - the non-saturated models where I define my first- and second-order variables are much different and do not show the same, expected results (a curious thing to me).

The first reason that comes to mind would be that your measurement model is misspecified, which casts doubt on the validity of the results using scale composites that assume your measurement model is correct.  MSEM also requires very large samples, and after listwise deletion, it appears you only have around 500 or so in your analysis.  That introduces a lot of sampling error, which (in addition to measurement error) could account for differences in your results.

[Andreas G.]

thank you very much for shedding light on this issue.

I cannot overstate how helpful your comments have been, I highly appreciate you taking your time for this!

Best regards,

Andreas