Question about pooling methods

[ahmad]

Hi,

I was wondering why I receive the following message:

“pool.robust = TRUE is only applicable when pool.method = 'D2'. Changed to pool.method = 'D2'.”

This occurs when I use the MLR estimator with pool.robust = TRUE in summary(). However, in the output it is stated that the D4 method was used (please see below). I am confused about why D4 is reported when the message indicates that D2 was applied.

My second question concerns which method is more appropriate to use:

• pool.robust = TRUE, which results in pooled.statistic = "yuan.bentler.mplus", or

• pool.robust = FALSE, which results in pooled.statistic = "standard".

I would appreciate any clarification on this.

summary(model.fit, standardized = TRUE, fit.measures = TRUE, rsq = TRUE, asymptotic = TRUE, pool.robust = TRUE,omit.imps = c("no.conv", "no.se", "no.npd")) pool.robust = TRUE is only applicable when pool.method = "D2". Changed to pool.method = "D2". lavaan.mi object fit to 100 imputed data sets using: - lavaan (0.6-21) - lavaan.mi (0.1-0) See class?lavaan.mi help page for available methods. Convergence information: The model converged on 100 imputed data sets. Standard errors were available for all imputations. Estimator ML Optimization method NLMINB Number of model parameters 52 Number of observations 4381 Model Test User Model: Standard Scaled Test statistic 126.511 123.631 Degrees of freedom 83 83 P-value 0.001 0.003 Average scaling correction factor 1.278 Pooling method D4 Pooled statistic “yuan.bentler.mplus” “yuan.bentler.mplus” correction applied AFTER pooling

Best wishes,

Ahmad

[Terrence Jorgensen]

I am confused about why D4 is reported when the message indicates that D2 was applied.

That is unexpected.  Did you notice that the pooled statistics are identical, whether you explicitly set pool.method="D2" or not?  I can verify that in fact, you are getting the D2 method, as the warning message indicates.  However, for some reason that change is not being tracked in the summary() method to print the correct method name as D2 (unless it is explicitly requested).  I'll try to track down that bug soon. 

My second question concerns which method is more appropriate to use:

• pool.robust = TRUE, which results in pooled.statistic = "yuan.bentler.mplus", or

• pool.robust = FALSE, which results in pooled.statistic = "standard".

Generally, I would not recommend using MLR.  The whole point of imputation is that you can use complete-data methods for the analysis stage.  MLM is much more stable than MLR in finite samples, but they are asymptotically equivalent, and I think you can consider your sample to be asymptotically large.

But your question is about whether to pool the robust statistic, or to pool the standard statistic which is then scaled to be robust.  The simulations I have seen show that pooling robust statistics doesn't work as well as "robustifying" pooled standard statistics, which is why that is the default.  See recent work by Fan Jia for nonnormal data (e.g., Figure 3).

 summary(model.fit, standardized = TRUE, fit.measures = TRUE, rsq = TRUE, asymptotic = TRUE, pool.robust = TRUE,omit.imps = c("no.conv", "no.se", "no.npd")) pool.robust = TRUE is only applicable when pool.method = "D2". Changed to pool.method = "D2". lavaan.mi object fit to 100 imputed data sets using: - lavaan (0.6-21) - lavaan.mi (0.1-0) See class?lavaan.mi help page for available methods. Convergence information: The model converged on 100 imputed data sets. Standard errors were available for all imputations. Estimator ML Optimization method NLMINB Number of model parameters 52 Number of observations 4381 Model Test User Model: Standard Scaled Test statistic 126.511 123.631 Degrees of freedom 83 83 P-value 0.001 0.003 Average scaling correction factor 1.278 Pooling method D4 Pooled statistic “yuan.bentler.mplus” “yuan.bentler.mplus” correction applied AFTER pooling

This is not easy to read.  I'm not sure how you are copy/pasting the Console output, but it appears to be pasted as a single long character string.  When you post output, please take the time to make sure it is formatted like in the Console, so it is readable.

Terrence D. Jorgensen    (he, him, his)
Assistant Professor, Methods and Statistics
Research Institute for Child Development and Education, the University of Amsterdam
http://www.uva.nl/profile/t.d.jorgensen

[Terrence Jorgensen]

I was wondering why I receive the following message:

“pool.robust = TRUE is only applicable when pool.method = 'D2'. Changed to pool.method = 'D2'.”

This is fixed in the development version.

remotes::install_github("TDJorgensen/lavaan.mi")

[ahmad valikhani]

Thank you, Dr.  Jorgensen.

I have tried the development version, but it seems the issue still persists; or I may be misunderstanding something. I am using ML-related estimators, but I am still getting the D2 pooling method. I have attached the results from both the CRAN version (which appears to have a bug) and the development version.

This is the result obtained using the CRAN version. Although it gave a warning that the pooling method changed to “D2,” the results section correctly reports the D4 pooling method. 

 

The following results were obtained from the development version, and I still receive the same message. The results are largely the same; however, the pooling method is specified as “D2” in the output. I did not specify pool.method = "D2", and since I am using an ML-related estimator, I would expect it to be “D4.”

 

Best wishes,

A

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/lavaan/a7d2294c-2268-4934-8a72-cfb46dc434d1n%40googlegroups.com.

[ahmad valikhani]

Sorry, regarding your question, " Did you notice that the pooled statistics are identical, whether you explicitly set pool.method="D2" or not?". When I specified pool.method="D2" explicitly, the pooled statistics are identical. However, when I specified  pool.method="D4" etc. I get the same warning and the results are identical to pool.method="D2" while in the output this appears: 

[Terrence Jorgensen]

I have tried the development version

Not according to the output you posted, which still states that lavaan.mi 0.1-0 as used.  If you successfully installed the development version (0.1-1.0032) from GitHub, you would see the issue is resolved:

HS.model <- '
  visual  =~ x1 + x2 + x3
  textual =~ x4 + x5 + x6
  speed   =~ x7 + x8 + x9
'

## fit model to imputed data sets
fit <- cfa.mi(HS.model, data = HS20imps, estimator = "MLM")
summary(fit, pool.robust = TRUE)

Output:

pool.robust = TRUE is only applicable when pool.method = "D2". Changed to pool.method = "D2".

lavaan.mi object fit to 20 imputed data sets using:
 - lavaan    (0.6-22.2551)
 - lavaan.mi (0.1-1.0032)

See class?lavaan.mi help page for available methods.

Convergence information:

The model converged on 20 imputed data sets.

Standard errors were available for all imputations.

  Estimator                                         ML
  Optimization method                           NLMINB

  Number of model parameters                        21

  Number of observations                           301

Model Test User Model:

                                                  Standard      Scaled

  Test statistic                                    64.253      63.298
  Degrees of freedom                                    24          24
  P-value                                            0.000       0.000
  Average scaling correction factor                              1.043
  Pooling method                                        D2            
    Pooled statistic                     “satorra.bentler”            
    “satorra.bentler” correction applied            BEFORE     pooling

If you thought you installed it, I recommend you try again in a fresh R session, then restart R again after installation completes.  Pay attention to whether any errors printed.  If it installed successfully, you should see output similar to this:

* installing *source* package ‘lavaan.mi’ ...
** this is package ‘lavaan.mi’ version ‘0.1-1.0032’
** using staged installation
** R
** data
*** moving datasets to lazyload DB
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (lavaan.mi)