Significance for model-implicit correlations of latent variables

[Xavier Suriol]

Dear lavaan users,

I would like to publish the model-implied correlation matrix of the latent variables with the significance (p_value) of each pair but this is not provided. I am running lavInspect(results, "cor.lv") (or lavInspect(results, "cov.lv")).

Do you know how to get it?  How do you publish model-implied correlations?

Can I run the function r.test() from R-package "psych"? It tests the significance of a single correlation.

I would appreciate any suggestions.

Thank you very much in advance,

Xavier Suriol

[Christian Arnold]

Hi,

This question has been asked probably a million times. Have you searched the forum? The (probably) most sensible approach is to bootstrap the correlations.

HTH

Christian 

---

From: lav...@googlegroups.com <lav...@googlegroups.com> on behalf of Xavier Suriol <xavier...@esade.edu>
Sent: Tuesday, May 2, 2023 10:02:09 PM
To: lavaan <lav...@googlegroups.com>
Subject: Significance for model-implicit correlations of latent variables

 

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/716f6a94-0b3e-4bd2-b36f-35c146a2b2dfn%40googlegroups.com.

[Xavier Suriol]

Thank you very much Christian for your quick response. I had glanced the forum... with no success because I used the keyword "implicit".

There are several solutions.

Regards,

Xavier Suriol

El dia dimarts, 2 de maig de 2023 a les 22:50:40 UTC+2, christia...@hhl.de va escriure:

[Christian Arnold]

Use x.boot or various other solutions. If you fit a CFA, you can easily define the correlations as parameters - also with the marker method. Take a look at standardizedSsolution. I think Shu Fai has provided a lot of input. You can use the delta method or better the bootstrap or MonteCarloCI.

---

From: lav...@googlegroups.com <lav...@googlegroups.com> on behalf of Xavier Suriol <xavier...@esade.edu>

Sent: Wednesday, May 3, 2023 12:36:21 AM
To: lavaan <lav...@googlegroups.com>
Subject: Re: Significance for model-implicit correlations of latent variables

 

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/d29766a1-6e14-40a8-b4dc-8882d74b0ad7n%40googlegroups.com.

[Shu Fai Cheung]

Christian is right that, for a CFA model, the correlations and their delta method CIs and p-values are already in the output of standardizedSolution.

Assuming that the model is not a CFA model, then whether it is easy to define the correlations using ":=" depends on the form of the model. Given that lavInspect() with "cor.lv" has been used, bootstrapLavaan() in lavaan (a very useful function) maybe the most straightforward solution:

library(lavaan)
#> This is lavaan 0.6-15
#> lavaan is FREE software! Please report any bugs.
# Create a dataset from a simple mediation model
# Use a large N such that the delta method CIs
# are good approximation.
n <- 10000
set.seed(8570614)
x <- rnorm(n)
m <- .3 * x + sqrt(1 - .3^2) * rnorm(n)
y <- .4 * m + sqrt(1 - .4^2) * rnorm(n)
es <- MASS::mvrnorm(n, rep(0, 9), diag(9))
tmp1 <- rep(.8, 3)
tmp2 <- rep(0, 3)
lambda <- matrix(c(tmp1, tmp2, tmp2,
                   tmp2, tmp1, tmp2,
                   tmp2, tmp2, tmp1), 9, 3)
xs <- cbind(x, m, y) %*% t(lambda) + es
dat <- setNames(as.data.frame(xs), paste0("x", 1:9))
head(dat)
#>           x1          x2          x3          x4         x5         x6
#> 1  0.7016253  0.06152504  0.68347512  2.02138386 -0.5280180  0.5149986
#> 2 -1.4035905 -0.99890373 -0.07430941 -0.59950393 -1.5518963  1.6910274
#> 3 -0.4406080 -2.39450424 -2.68569378 -1.24964144 -0.6139338 -0.4155945
#> 4 -0.2377625  1.45487535  1.69618648  0.08763255  1.6434675  0.4468465
#> 5  0.4607871 -0.43366503 -0.48382764 -0.17073499 -2.2080429 -0.8658210
#> 6 -1.7749543 -1.41704701 -0.43697121  1.23592253  0.7286826 -0.9211944
#>           x7          x8          x9
#> 1 -1.2952441  0.18894715 -0.30120103
#> 2 -0.9708784 -0.05597014  0.11817134
#> 3 -2.3909430 -1.20812274 -2.08156240
#> 4 -1.2341506  1.63384080 -0.78532321
#> 5 -1.0080350  0.79803321  0.02724673
#> 6 -2.1795875 -0.10557664 -0.60519642
mod <-
"
x =~ x1 + x2 + x3
m =~ x4 + x5 + x6
y =~ x7 + x8 + x9
m ~ x
y ~ m
"
fit <- sem(mod, dat)
lavInspect(fit, "cor.lv")
#>       x     m     y
#> x 1.000            
#> m 0.311 1.000      
#> y 0.129 0.415 1.000

# Get the unique elements
lav_matrix_vech(lavInspect(fit, "cor.lv"), diagonal = FALSE)
#> [1] 0.3113329 0.1292602 0.4151833
# Define a function for bootstrapLavaan()
cor_lv <- function(object) {
    out <- lavaan::lav_matrix_vech(lavaan::lavInspect(object, "cor.lv"),
                                   diagonal = FALSE)
    stats::setNames(out, c("x~~m", "x~~y", "m~~y"))
  }
# Check the function
cor_lv(fit)
#>      x~~m      x~~y      m~~y 
#> 0.3113329 0.1292602 0.4151833
lavInspect(fit, "cor.lv")
#>       x     m     y
#> x 1.000            
#> m 0.311 1.000      
#> y 0.129 0.415 1.000

# Do bootstrapping
boot_out <- bootstrapLavaan(fit,
                            R = 2000,
                            FUN = cor_lv,
                            parallel = c("snow"),
                            ncpus = 9,
                            iseed = 53243)
# The output
head(boot_out)
#>           x~~m      x~~y      m~~y
#> [1,] 0.3181989 0.1408866 0.4427627
#> [2,] 0.2961989 0.1221270 0.4123140
#> [3,] 0.3149348 0.1237163 0.3928313
#> [4,] 0.3311231 0.1387524 0.4190356
#> [5,] 0.3241540 0.1343073 0.4143318
#> [6,] 0.3240413 0.1373217 0.4237785
# Keep rows without NAs
boot_out <- na.omit(boot_out)
# Check the means of the bootstrap estimates
colMeans(boot_out)
#>      x~~m      x~~y      m~~y 
#> 0.3115700 0.1292970 0.4149475
cor_lv(fit)
#>      x~~m      x~~y      m~~y 
#> 0.3113329 0.1292602 0.4151833
# Form the 95% percentile CIs
t(apply(boot_out, 2, quantile, probs = c(.025, .975), na.rm = TRUE))
#>           2.5%     97.5%
#> x~~m 0.2842050 0.3385694
#> x~~y 0.1142739 0.1441069
#> m~~y 0.3866340 0.4437185
# Compare with delta method CIs
# For this model, correlations x~~m and m~~y
# are equal to the standardized m~x and y~m, respectively.
standardizedSolution(fit)[c(10:11), ]
#>    lhs op rhs est.std    se      z pvalue ci.lower ci.upper
#> 10   m  ~   x   0.311 0.014 21.918      0    0.283    0.339
#> 11   y  ~   m   0.415 0.014 30.619      0    0.389    0.442

The main function is this one, which can be easily adapted for other models:

cor_lv <- function(object) {
    out <- lavaan::lav_matrix_vech(lavaan::lavInspect(object, "cor.lv"),
                                   diagonal = FALSE)
    stats::setNames(out, c("x~~m", "x~~y", "m~~y"))
  }

This method does not give the p-values. I wish there is a function as simple and flexible as bootstrapLavaan() but for delta method SEs and CIs (and p-values, if necessary). However, I am not aware of one. There are situations in which this simple method can be justified, e.g., when the sample size is large.

Regards,

Shu Fai Cheung (張樹輝)

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/AM0PR04MB595528B5FACC3327EE371290FC6F9%40AM0PR04MB5955.eurprd04.prod.outlook.com.

[Xavier Suriol]

Thank you very much Shu Fai Cheung (張樹輝) for your function, and again I thank Christian because I have seen the code you provided on October 11^th 2022.

As my model is a CFA, I first looked for solutions in existing functions in lavaan but it is not so straight: parameterestimates() and standardizedSolution() are enough for models like the popular visual-textual-speed because all possible correlations within the model are already in “Parameter Estimates” (default output) but my model is a third-degree CFA and not all possible correlations are in “Parameter Estimates”. Let’s say that, as seen in a plot, two different subfactors may be in two unrelated corners.

lavInspect(results, "cor.lv") provides all possible pairs of correlations. Maybe standardizedSolution() provides all of them by using GLIST, est, and partable arguments but I disregarded this approach after reading the Note “The est, GLIST, and partable arguments are not meant for everyday users, but for authors of external R packages that depend on lavaan. Only to be used with great caution.”

At the end I ran the following, which is easier than your elaborated approaches (for instance, no lav_matrix() is used):

bootCorrelations <- bootstrapLavaan(results, R=2000,FUN=lavInspect, what="cor.lv")

And then the 95% CI will be at quantiles 2.5 and 97.5%

I would appreciate if you could confirm this is also correct. The means of the bootstrap are pretty similar to cor.lv correlations.

 

Again, thank you very much for your attention. I will appreciate any help.

Best regards,

Xavier Suriol

El dia dimecres, 3 de maig de 2023 a les 13:47:56 UTC+2, shufai...@gmail.com va escriure:

[Christian Arnold]

Hi Xavier,

Sounds good at first sight. If you want to compute a different confidence interval (for example bias corrected), then feel free to copy the CI function from x.boot and pass t0 (the estimate for the correlation), t (the estimates of the bootstrap draws for the cor), the CI level and the desired CI interval. Ideally this should work without problems.

Best

Christian 

---

Von: lav...@googlegroups.com <lav...@googlegroups.com> im Auftrag von Xavier Suriol <xavier...@esade.edu>
Gesendet: Mittwoch, 3. Mai 2023, 21:47
An: lavaan <lav...@googlegroups.com>
Betreff: Re: Significance for model-implicit correlations of latent variables

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/d8898629-6afb-4360-9bcc-1e9831ee8188n%40googlegroups.com.

Christian 

[Shu Fai Cheung]

Thanks, Xavier! Your method works! I learned something too!

Just for checking, this is a version of the example using your method, showing that the results are exactly identical to those using the function I wrote (they should be):

boot_out_direct <- bootstrapLavaan(fit,
                            R = 2000,
                            FUN = lavInspect,

                            parallel = c("snow"),
                            ncpus = 9,

                            iseed = 53243,
                            what = "cor.lv")
# The output
head(boot_out_direct)
#>      [,1]      [,2]      [,3]      [,4] [,5]      [,6]      [,7]      [,8] [,9]
#> [1,]    1 0.3181989 0.1408866 0.3181989    1 0.4427627 0.1408866 0.4427627    1
#> [2,]    1 0.2961989 0.1221270 0.2961989    1 0.4123140 0.1221270 0.4123140    1
#> [3,]    1 0.3149348 0.1237163 0.3149348    1 0.3928313 0.1237163 0.3928313    1
#> [4,]    1 0.3311231 0.1387524 0.3311231    1 0.4190356 0.1387524 0.4190356    1
#> [5,]    1 0.3241540 0.1343073 0.3241540    1 0.4143318 0.1343073 0.4143318    1
#> [6,]    1 0.3240413 0.1373217 0.3240413    1 0.4237785 0.1373217 0.4237785    1

# Keep rows without NAs

boot_out_direct <- na.omit(boot_out_direct)

# Check the means of the bootstrap estimates

colMeans(boot_out_direct)
#> [1] 1.0000000 0.3115700 0.1292970 0.3115700 1.0000000 0.4149475 0.1292970
#> [8] 0.4149475 1.0000000

cor_lv(fit)
#>      x~~m      x~~y      m~~y 
#> 0.3113329 0.1292602 0.4151833
# Form the 95% percentile CIs

t(apply(boot_out_direct, 2, quantile, probs = c(.025, .975), na.rm = TRUE))
#>            2.5%     97.5%
#>  [1,] 1.0000000 1.0000000
#>  [2,] 0.2842050 0.3385694
#>  [3,] 0.1142739 0.1441069
#>  [4,] 0.2842050 0.3385694
#>  [5,] 1.0000000 1.0000000
#>  [6,] 0.3866340 0.4437185
#>  [7,] 0.1142739 0.1441069
#>  [8,] 0.3866340 0.4437185
#>  [9,] 1.0000000 1.0000000

# Compare with delta method CIs
# For this model, correlations x~~m and m~~y
# are equal to the standardized m~x and y~m, respectively.
standardizedSolution(fit)[c(10:11), ]
#>    lhs op rhs est.std    se      z pvalue ci.lower ci.upper
#> 10   m  ~   x   0.311 0.014 21.918      0    0.283    0.339
#> 11   y  ~   m   0.415 0.014 30.619      0    0.389    0.442

Although the documentation of bootstrapLavaan() states that FUN is expected to be a function that returns a vector, your method works because the output of lavInspect with what = "cor.lv", though a matrix, is converted to a vector internally.

An interesting and useful technique! Thanks!

Regards,

Shu Fai

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/d8898629-6afb-4360-9bcc-1e9831ee8188n%40googlegroups.com.

[Xavier Suriol]

Thanks to you, Shu Fai. We all know a shorter script now. Your recent verification was necessary. Thanks.

Thank you, Christian, for your x.boot script that I will certainly try. I will let you know.

Regards,

Xavier

El dia dijous, 4 de maig de 2023 a les 0:24:18 UTC+2, shufai...@gmail.com va escriure:

[Xavier Suriol]

Hi Christian,

I ran your x.boot script but the execution failed because "Error in FUN (object,...) : unused argument (return.boot = FALSE)".

Do you think this can be fixed?

Thank you very much in advance.

Best regards,

Xavier

El dia dijous, 4 de maig de 2023 a les 17:24:04 UTC+2, Xavier Suriol va escriure:

[Xavier Suriol]

Hi Christian, 

Your latest release of x.boot worked fine. Thank you very much indeed.

Do you know how to cite a script released in a google group forum? Is the following correct?

Arnold, Christian (2023, May 6).  “Bootstrap estimates, standardized solution, R squares, HTMT, group differences, consistent p-values, and other coefficients with a single bootstrap run” [Online forum script release]. Message posted to https://groups.google.com/g/lavaan/c/0RSsh4M6zQg

I was inspired by https://utas.libguides.com/c.php?g=498348&p=3412833

Best regards,

Xavier

El dia divendres, 5 de maig de 2023 a les 14:52:35 UTC+2, Xavier Suriol va escriure:

[Christian Arnold]

Hi Xavier,

I don't have a perfect answer. x.boot is intended as a help and discussion. I don't know if reviewers accept it. Sorry ...I think x.boot gives decent results. You can compare it with lavaan (parameterEstimates/standardizedSolution .... and boot.ci.type). It would be nice to get feedback from other users and experts. That's the charm of open source 😁

Best

Christian 

---

Von: lav...@googlegroups.com <lav...@googlegroups.com> im Auftrag von Xavier Suriol <xavier...@esade.edu>

Gesendet: Sonntag, 7. Mai 2023, 22:27
An: lavaan <lav...@googlegroups.com>
Betreff: Re: Significance for model-implicit correlations of latent variables

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/0f37114a-c4d2-44b4-a408-08ebe3ee64f4n%40googlegroups.com.