Convergence issues on BSEM with deterministic missingness

69 views
Skip to first unread message

Praveen Nair

unread,
Nov 30, 2023, 10:51:58 PM11/30/23
to blavaan

Hello everyone,

I’m a beginner to Bayesian latent variable modeling, and I’ve been trying to use blavaan for a research project. Currently, as a proof of concept, I’m fitting a linear-Gaussian SEM on a synthetic dataset, where we have two features X_1 and X_2, one latent variable C, and two outcome variables D and Y. The values for the dataset are generated from a given SEM.

One really important point to note is that there is deterministic missingness in Y: if the value of D is 0 in the dataset, then Y is missing. (But D has no other direct effect on Y.) Currently, I am using the default FIML method to handle missingness, but perhaps FIML isn’t built for this situation?

The following plot is the SEM that I’m trying to estimate.

sem.png

Below are plots of the values in this synthetic dataset, with the points’ position given by X, and colored by D in the first plot, and Y in the second. (Note that due to deterministic missingness, all the points in the graph colored by Y come from the subset of points where D = 1.)

D_graph.pngY_graph.png

The point is: the model takes longer than I believe it should to converge, often using over 10,000 iterations to converge (per the blavaan documentation, “If your model is not converging after 1,000 burnin samples, it is likely that the default prior distributions clash with your data.”) I’ve tried tightening the priors, and while that hasn’t improved convergence, I haven’t messed with the priors in any very structured way, so perhaps that is still the solution.

I was hoping someone could advise me on whether I’m making any mistakes using the package, or whether this is just inherent to the problem I'm trying to specify. The following is the model I’m using to specify the model, and the code I’m running to estimate it in blavaan.

# import dataset
data <- read.csv('data/dataset_current.csv', header = TRUE)


# create lavaan model
model <- '
  C =~ NA*X_1 + X_2 + D + Y
  D ~ X_1 + X_2
 
  # everything has variance
  C ~~ C
  D ~~ D
  Y ~~ Y
  X_1 ~~ X_1
  X_2 ~~ X_2
 
  # correlations between features
  X_1 ~~ X_2
 
  # and everything has an intercept
  C ~ 1
  D ~ 1
  Y ~ 1
  X_1 ~ 1
  X_2 ~ 1
'

BURN <- 1000
rhat <- 20
while(rhat > 1.01) {
  BURN <- BURN + 1000
  fit_continuous <- bsem(model, data = data, n.chains = 4, sample = 1000, burnin = BURN, bcontrol = list(cores = 4), mcmcfile = TRUE, save.lvs = TRUE)
  rhat <- max(blavInspect(fit_continuous, "psrf"), na.rm = TRUE)
  print(paste0("Rhat=", rhat))
}

Thank you in advance for your help, and for the work and support y'all provide in creating and maintaining a solution like blavaan!

Mauricio Garnier-Villarreal

unread,
Dec 1, 2023, 4:29:04 AM12/1/23
to blavaan
Hi

could you show the summary() of your model? I think you might have an unidentified model because you relaease the constraint on the first factor loading ( C =~ NA*X_1 + X_2 + D + Y ), but didnt add any other identification method. Try adding the argument std.lv=T to the bsem function to identofy it by fixing the latent variance to 1

If that still doesnt work, we can look at using the priors to constraint the parameter space.

About the missing structucture. These missing patterns are notoriously problemaic. If you use the default FIMCMC (full information MCMC, is what I call the approach used in blavaan as it is similar to FIML but is not ML) would estimate the model as if everyone had answer everything to estimate the parameters. But would lead to interpretatibility issues

Praveen Nair

unread,
Dec 2, 2023, 12:10:42 PM12/2/23
to blavaan
Thanks for the response! I tried setting std.lv to TRUE, and removing the NA from the definition of C, but that hasn't seemed to solve things. I tried running bsem() with 8000 burn-in iterations and 1000 samples, and the model doesn't converge -- the summary() output is below. I imagine the most likely problem is with the missingness pattern, then, but any other ideas are appreciated. (Or how to work with the priors -- should I just make them tighter around 0?)

blavaan 0.5.2 did NOT end normally after 1000 iterations
** WARNING ** Estimates below are most likely unreliable

  Estimator                                      BAYES
  Optimization method                             MCMC
  Number of model parameters                        16

  Number of observations                          1000
  Number of missing patterns                         2

  Statistic                                 MargLogLik         PPP
  Value                                             NA       0.542

Parameter Estimates:


Latent Variables:
                   Estimate  Post.SD pi.lower pi.upper     Rhat    Prior      
  C =~                                                                        
    X_1               1.961    0.139    1.585    2.127    1.098    normal(0,10)
    X_2               1.474    0.142    1.138    1.699    1.183    normal(0,10)
    D                -0.699    1.811   -4.410    2.936    1.279    normal(0,10)
    Y                 0.220    0.064    0.143    0.379    1.118    normal(0,10)

Regressions:
                   Estimate  Post.SD pi.lower pi.upper     Rhat    Prior      
  D ~                                                                          
    X_1               0.379    0.658   -1.029    1.667    1.248    normal(0,10)
    X_2               0.182    0.346   -0.439    1.155    1.293    normal(0,10)

Covariances:
                   Estimate  Post.SD pi.lower pi.upper     Rhat    Prior      
 .X_1 ~~                                                                      
   .X_2              -0.194    0.349   -0.548    0.803    1.191       beta(1,1)

Intercepts:
                   Estimate  Post.SD pi.lower pi.upper     Rhat    Prior      
    C                 6.627    3.437   -1.159   12.514    1.304    normal(0,10)
   .D                -0.839    4.736  -10.380    9.852    1.131    normal(0,10)
   .Y                -0.739    0.745   -2.113    0.890    1.206    normal(0,32)
   .X_1              -3.067    6.905  -15.388   12.203    1.297    normal(0,10)
   .X_2              10.107    5.290    1.099   21.581    1.314    normal(0,10)

Variances:
                   Estimate  Post.SD pi.lower pi.upper     Rhat    Prior      
    C                 1.000                                                    
   .D                 0.061    0.042    0.000    0.120    1.093 gamma(1,.5)[sd]
   .Y                 0.143    0.016    0.122    0.163    1.078 gamma(1,.5)[sd]
   .X_1               0.494    0.459    0.044    1.824    1.105 gamma(1,.5)[sd]
   .X_2               1.172    0.379    0.527    2.010    1.195 gamma(1,.5)[sd]

Ed Merkle

unread,
Dec 2, 2023, 12:43:36 PM12/2/23
to Praveen Nair, blavaan
I am not sure that the missingness thing is the major problem. I think the parameters of this model are not identified, beyond std.lv=TRUE or FALSE. All the observed variables are connected by regression relationships (directed paths), and also by the latent variable C. So I think that the regression parameters will trade off with the loadings (the paths coming off C), which would explain why the model is not converging.

This is a place where lavaan can really help, because it will quickly tell you something is wrong in the maximum likelihood version of the model. Just do

library("lavaan")
sem(model, data = data)

For this model, I believe lavaan will report negative degrees of freedom and also report that it can't invert the information matrix. I think this means that you need to either eliminate free parameters or constrain some parameters to be equal. Mauricio also mentioned using informative priors to try to constrain the parameters to an identified posterior distribution, which may be possible but I think also tricky.

Best,
Ed
Hi

could you show the summary() of your model? I think you might have an unidentified model because you relaease the constraint on the first factor loading (C =~ NA*X_1 + X_2 + D + Y), but didnt add any other identification method. Try adding the argument std.lv=T to the bsem function to identofy it by fixing the latent variance to 1

If that still doesnt work, we can look at using the priors to constraint the parameter space.

About the missing structucture. These missing patterns are notoriously problemaic. If you use the default FIMCMC (full information MCMC, is what I call the approach used in blavaan as it is similar to FIML but is not ML) would estimate the model as if everyone had answer everything to estimate the parameters. But would lead to interpretatibility issues



On Friday, December 1, 2023 at 4:51:58 AM UTC+1 prn...@ucsd.edu wrote:
Hello everyone,
I’m a beginner to Bayesian latent variable modeling, and I’ve been trying to use blavaan for a research project. Currently, as a proof of concept, I’m fitting a linear-Gaussian SEM on a synthetic dataset, where we have two features X_1 and X_2, one latent variable C, and two outcome variables D and Y. The values for the dataset are generated from a given SEM.
One really important point to note is that there is deterministic missingness in Y: if the value of D is 0 in the dataset, then Y is missing. (But D has no other direct effect on Y.) Currently, I am using the default FIML method to handle missingness, but perhaps FIML isn’t built for this situation?
The following plot is the SEM that I’m trying to estimate.
sem.png
Below are plots of the values in this synthetic dataset, with the points’ position given by X, and colored by D in the first plot, and Y in the second. (Note that due to deterministic missingness, all the points in the graph colored by Y come from the subset of points where D = 1.)
D_graph.pngY_graph.png
--
You received this message because you are subscribed to the Google Groups "blavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blavaan+u...@googlegroups.com.

Mauricio Garnier-Villarreal

unread,
Dec 3, 2023, 6:49:57 PM12/3/23
to blavaan
With 4 variables, you have 10 unique elements of the coavriance matrix, and have 11 parameters (means dont count in this). So, need to remove at least 1 parameter

Also, another problem is that even with std.lv=T, the mean on C is being estimated because you wrote it in the sytnax, instead of being fixed to 0. Now, it might be that having the variables have direct relations, and relations through the latent variable could make the model hard to find a solution

I would simplify the syntax like this, and rely on the defaults to estimate the variances (by default) and the means (meanstructure=T).

Then need to remove at least one parameter

# create lavaan model
model <- '
  C =~ X_1 + X_2 + D + Y

  D ~ X_1 + X_2
 
  # correlations between features
  X_1 ~~ X_2

'

BURN <- 1000
rhat <- 20
while(rhat > 1.01) {
  BURN <- BURN + 1000
  fit_continuous <- bsem(model, data = data, std.lv=T, meanstructure=T,
n.chains = 4, sample = 1000, burnin = BURN, 
bcontrol = list(cores = 4), mcmcfile = TRUE, save.lvs = TRUE)

  rhat <- max(blavInspect(fit_continuous, "psrf"), na.rm = TRUE)
  print(paste0("Rhat=", rhat))
}

Praveen Nair

unread,
Dec 4, 2023, 5:44:48 PM12/4/23
to blavaan
Thanks! I've simplified the model as you specified, and since the particular problem I'm working on allows this, I've fixed the loading from C to Y to be 1. The convergence is definitely much better -- the R-hat is 1.011 after only 2,000 burn-in iterations, although it takes some more iterations for effective sample size to reach the recommended levels. Still, this is a massive improvement. And I see what you mean about the model being hard to estimate -- I'll think about ways to mitigate this, because in practice, I plan to have the MCMC samples from the blavaan estimation be part of an active learning loop, so I'll be running the same estimation on slightly different datasets iteratively.
Reply all
Reply to author
Forward
0 new messages