Model Doesn't Converge with 0.6-4 Release

[Andrew Grotzinger]

Hello, 

I've been running models in lavaan with user-provided covariance and sampling covariance matrices using DWLS estimation and am finding that models that used to converge using release 0.6-3 are no longer converging using release 0.6-4. I did not see anything posted in that version history that indicated updates that should affect these particular models, and was wondering if you could provide any insight as to why the update has affected model convergence? Thanks very much!

#load in the data

load("Data.RData")

##load prior version of lavaan

library(devtools)

install_version("lavaan", version = "0.6-3", repos = "http://cran.us.r-project.org")

require(lavaan)

Model1<-"F1=~NA*V1 + V2 + V3 + V4 + V5

F1~~1*F1"

Test<-sem(Model1, sample.cov = Data[[1]], estimator = "DWLS", WLS.V = Data[[2]], sample.nobs = 2)

summary(Test)

##restart R and load 0.6-4 version of lavaan

install.packages("lavaan")

require(lavaan)

Test2<-sem(Model1, sample.cov = Data[[1]], estimator = "DWLS", WLS.V = Data[[2]], sample.nobs = 2)

summary(Test2)

[Andrew Grotzinger]

Here is the data I used for the example code as an .RData object.

[Yves Rosseel]

On 7/8/19 10:17 PM, Andrew Grotzinger wrote:
> Hello,
>
> I've been running models in lavaan with user-provided covariance and
> sampling covariance matrices using DWLS estimation and am finding that
> models that used to converge using release 0.6-3 are no longer
> converging using release 0.6-4.

There is indeed a subtle change: 0.6-4 has an additional check to see if
all the elements of the gradient are (near) zero. If not, it sets
converged = FALSE. In 99.9% of the cases, this additional check is
harmless. You were unlucky.

Currently, the check uses 0.0001 as the tolerance value. It turns out
that in your example, you have a few gradient elements just above
0.0001. To 'make' it converge, you can use

Test<-sem(Model1, sample.cov = Data[[1]], estimator = "DWLS", WLS.V =

Data[[2]], sample.nobs = 2, optim.dx.tol = 0.001)

To switch off this test altogether, you can use

Test<-sem(Model1, sample.cov = Data[[1]], estimator = "DWLS", WLS.V =

Data[[2]], sample.nobs = 2, optim.dx.tol = +Inf)

The reason why you have trouble is the scaling. 0.6-4 currently looks at
the unscaled gradient elements only. In a future update, I will also
'scale' some problematic elements to avoid false positives.

Finding a good balance to determine 'true' or 'false' convergence is
somewhat of an art.

> I did not see anything posted in that
> version history that indicated updates that should affect these
> particular models

You are right. This is an oversight on my part. I have now updated the
release notes.

Yves.

[Andrew Grotzinger]

Hi Yves, 

Thanks very much for your quick reply and help! Setting the tolerance value to a much smaller number is working in most cases. I'm finding that there is still an instance where 0.6-3 will run but 0.6-4 prints an error even when using optim.dx.tol = +Inf. The model we are running is admittedly a bit strange, but we use it for follow-up models. Do you have any additional suggestions beyond the tolerance that would explain why this particular model might not run with 0.6-4, and how we might change the arguments so that it runs as with 0.6-3? The file Data2.RData is attached to this post. Thanks again!

Model1<-"F1 =~ NA*V1 + V2 + V3 + V4 + V5

F1 ~~ 1*F1

VF1 =~ 1*V1

VF1 ~~ 0*VF1

V1 ~~ V1

VF2 =~ 1*V2

VF2 ~~ 0*VF2

V2 ~~ V2

VF1~~ 0*VF2

F1 =~ 0*VF1 + 0*VF2"

load("Data2.RData")

require(lavaan)

Model1a <- sem(Model1, sample.cov = Data2[[1]], estimator = "DWLS", WLS.V = Data2[[2]], sample.nobs = 2, optim.dx.tol = +Inf)

library(devtools)

install_version("lavaan", version = "0.6-3", repos = "http://cran.us.r-project.org")

require(lavaan)

Model1b <- sem(Model1, sample.cov = Data2[[1]], estimator = "DWLS", WLS.V = Data2[[2]], sample.nobs = 2) 

[Yves Rosseel]

On 7/9/19 8:11 PM, Andrew Grotzinger wrote:
> Hi Yves,
>
> Thanks very much for your quick reply and help! Setting the tolerance
> value to a much smaller number is working in most cases. I'm finding
> that there is still an instance where 0.6-3 will run but 0.6-4 prints an
> error even when using optim.dx.tol = +Inf. The model we are running is
> admittedly a bit strange, but we use it for follow-up models.

This one has nothing to do with the additional check. Run both models
with verbose = TRUE, and you will see it fails in 0.6-4.

It has to do with poor starting values. This is an atypical model, and
therefore, the automatically generated starting values for the factor
loadings are just '1' for all loadings. For this model, this turns out
to be a poor choice.

In 0.6-4, I added a check to make sure that the diagonal elements of the
model-implied covariance matrix are always positive. For some reason,
after three or four iterations, the parameters in this model do generate
negative diagonal variances. In 0.6-3, this was ignored (hoping for the
best), sometimes leading to good results, but also sometimes leading to
strange results. In 0.6-4, the objective value is now 'NA', prompting
the optimizer to change course. Usually, this works better, but in this
case, the optimizer does not recover.

Providing better starting values helps. For example:

Model1<-"F1 =~ NA*V1 + start(0.1)*V2 + V3 + V4 + V5

F1 ~~ 1*F1
VF1 =~ 1*V1
VF1 ~~ 0*VF1
V1 ~~ V1
VF2 =~ 1*V2
VF2 ~~ 0*VF2
V2 ~~ V2
VF1~~ 0*VF2
F1 =~ 0*VF1 + 0*VF2"

seems to work.

Alternatively, as 'V4' was the offending variable, this will also work
(in 0.6-4):

Model1<-"F1 =~ NA*V1 + V2 + V3 + V4 + V5

# 'force' residual variance to be positive
V4 ~~ lower(0)*V4

F1 ~~ 1*F1
VF1 =~ 1*V1
VF1 ~~ 0*VF1
V1 ~~ V1
VF2 =~ 1*V2
VF2 ~~ 0*VF2
V2 ~~ V2
VF1~~ 0*VF2
F1 =~ 0*VF1 + 0*VF2"

Yves.

[Andrew Grotzinger]

Hi Yves, 

This is very helpful, thanks so much for troubleshooting!

Best, 

  Andrew