huge modificationindices values + interpretations

1,576 views

Skip to first unread message

luggie müller

unread,

Apr 15, 2018, 7:38:21 AM4/15/18

to lavaan

Hi,
got the following, quite well fitting

                                                DWLS      Robust
Minimum Function Test Statistic               10.740      30.338
Degrees of freedom                                28          28
P-value (Chi-square)                           0.999       0.347

Comparative Fit Index (CFI)                    1.000       0.987
Tucker-Lewis Index (TLI)                       1.079       0.975
RMSEA                                          0.000       0.035
SRMR                                           0.071       0.071

model

model_10_4 <- '
#latent variables
management =~ ForMI+SMI
biodiv =~ shannon_herbs + Simpsons_herbs
tree_type =~ mts
light =~ mean_LEC_corr+H_ell_weighted
soil =~ N_stock+Sand
trees =~ B2
herbs =~ H_fs

#regressions
herbs ~ d*management+ e*soil +h*light
tree_type ~ a*management+ g*soil       
trees ~ b*tree_type+soil+management
light ~ c*trees              +tree_type
biodiv ~ f*herbs             +management+soil

#mediator effect 
#ab := a*b
#bc := c*b
#abc := c*b*a
#ef := e*f
#df := d*f
#bg := b*g
#hf := h*f


#co(variances)
Sand ~~ mean_LEC_corr +B2
N_stock ~~ mean_LEC_corr +B2
'

that explains the variance (R^2) of my dependent variables to agree, that I'm happy with it.

biodiv            0.682
tree_type         0.825
light             0.757
trees             0.800
herbs             0.438

Still, of course I'd like to improve it. Therefore I tried using modification indices,
Calling the functions puts some warnings messages, that I don't know how to interpret:

Warning messages:
1: In lav_start_check_cov(lavpartable = lavpartable, start = START) :
  lavaan WARNING: starting values imply a correlation larger than 1;
                  variables involved are: mean_LEC_corr Sand
2: In lav_start_check_cov(lavpartable = lavpartable, start = START) :
  lavaan WARNING: non-zero covariance element set to zero, due to fixed-to-zero variances
                  variables involved are: Sand B2
3: In lav_start_check_cov(lavpartable = lavpartable, start = START) :
  lavaan WARNING: non-zero covariance element set to zero, due to fixed-to-zero variances
                  variables involved are: N_stock B2

Also the MI score values are extremely huge. What could I read from this?

> head(mod_ind[order(mod_ind$mi, decreasing=TRUE), ], 20)
           lhs op            rhs        mi mi.scaled       epc   sepc.lv  sepc.all  sepc.nox
227 management  ~         biodiv 17191.673 30840.887 -8479.085 -8628.931 -8628.931 -8628.931
233       soil  ~         biodiv  8089.098 14511.382   407.889  3756.640  3756.640  3756.640
211  tree_type  ~          light  2478.179  4445.712   860.656   136.176   136.176   136.176
214      trees  ~          light  2470.982  4432.801  -646.337  -274.779  -274.779  -274.779
104      light =~             B2  2455.567  4405.148  -644.318  -104.774  -273.921  -273.921
232       soil  ~          light   235.286   422.090    -4.365   -11.013   -11.013   -11.013
229       soil  ~          herbs   164.071   294.334     3.663    24.362    24.362    24.362
113       soil =~             B2    80.328   144.105    18.312     1.180     3.086     3.086
122      trees =~        N_stock    69.748   125.124     0.514     0.197     2.494     2.494
209  tree_type  ~          herbs    51.873    93.058    -8.326    -3.472    -3.472    -3.472
212  tree_type  ~         biodiv    41.683    74.776    -3.355    -1.938    -1.938    -1.938
215      trees  ~         biodiv    34.384    61.683     2.114     3.281     3.281     3.281
213      trees  ~          herbs    29.974    53.772     3.156     3.536     3.536     3.536
223 management  ~          herbs    18.164    32.585   -17.381   -12.772   -12.772   -12.772
102      light =~        N_stock    15.200    27.267    -0.347    -0.056    -0.716    -0.716
134      herbs =~             B2     8.508    15.262     0.794     0.340     0.890     0.890
85      biodiv =~             B2     6.634    11.901     0.345     0.205     0.536     0.536
93   tree_type =~        N_stock     5.126     9.196    -0.025    -0.026    -0.325    -0.325
226 management  ~          light     4.853     8.706     8.939     2.492     2.492     2.492
100      light =~ Simpsons_herbs     3.313     5.944    -0.218    -0.035    -0.166    -0.166

Most of the suggested changes make no sense in theory but would according to the table impact the model a lot.
Can I conclude anything from this?

Thanks
luggie

Terrence Jorgensen

unread,

Apr 17, 2018, 5:38:07 AM4/17/18

to lavaan

warnings messages, that I don't know how to interpret:

Warning messages:
1: In lav_start_check_cov(lavpartable = lavpartable, start = START) :
  lavaan WARNING: starting values imply a correlation larger than 1;
                  variables involved are: mean_LEC_corr Sand

Look at your standardized estimates

summary(fit, std = TRUE)

Is that correlation larger than 1? That is what the warning is telling you. If you have an impossible estimate, then modification indices (or EPCs) based on that solution might not make sense.

2: In lav_start_check_cov(lavpartable = lavpartable, start = START) :
  lavaan WARNING: non-zero covariance element set to zero, due to fixed-to-zero variances
                  variables involved are: Sand B2
3: In lav_start_check_cov(lavpartable = lavpartable, start = START) :
  lavaan WARNING: non-zero covariance element set to zero, due to fixed-to-zero variances
                  variables involved are: N_stock B2

These refer to your labeled covariance (mean_LEC_corr ) between the indicators of soil and the single indicator of trees. Because B2 is a single indicator, its residual variance is set to zero and loading set to one, so all of its variance is represented at the latent level (trees). How can B2 have residual covariance if it does not have any residual variance? Also, I would expect empirical identification issues because you are estimating the relationship between soil and trees (as a regression) yet also estimating the relationship between their indicators. Why would that make sense?

Also the MI score values are extremely huge. What could I read from this? Most of the suggested changes make no sense in theory but would according to the table impact the model a lot.

You should not interpret modification indices as your data telling you to do anything. They are meant as a shortcut to substitute for a likelihood ratio test between nested models, without requiring any less-restricted models to be fit. You should only pay attention to ones that make sense to begin with. Exploration does not lead to generalizable results:

MacCallum, R. C., Roznowski, M., & Necowitz, L. B. (1992). Model modifications in covariance structure analysis: The problem of capitalization on chance. Psychological Bulletin, 111(3), 490.

I would recommend looking at correlation residuals

resid(fit, type = "cor")

They merely tell you which relationships are not well accounted for in the model (e.g., residuals > 0.1 in absolute value are considered large deviations between observed and expected values). That allows you to think about how your model can make up for that, rather than allowing modification indices to think for you. Modification indices assume your model is already correctly specified, except that a single additional parameter should be freely estimated. That is rarely a safe assumption.

Terrence D. Jorgensen

Postdoctoral Researcher, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam