reliability of p-values of standardizedSolution

[Janitor]

Hi,

I'm running several analysis with lavaan and encountered some strange p-value differences in my analysis. Using the summary(model...)-function, I do get a significant path with a p-value of .03. Looking at the standardizedSolution, however, the I get a p-value of ~.5 for the exact same path. 

Based on some of the discussions in this forum I did get the impression that p-values for standardized parameters using the stand.Sol.-function can be trusted. Somewhere else, I read that one should report standardized coefficients paired with p-values of the unstandardized solution. Thus, I'm a bit confused now and would appreciate any suggestions by the experts of this forum. 

Thanks in advance. 

Jan

[Patrick (Malone Quantitative)]

Jan,

Disclaimer: I haven't used stand.Sol.

Broadly speaking in SEM, the unstandardized solution should be used for hypothesis tests.

Consider that the unstandardized path coefficient is a single parameter estimate with its standard error.

The standardized path coefficient is that estimate, multiplied by another point estimate treated as an exact value, and divided by yet another point estimate treated as an exact value.

So the standardized estimate has much more room for deviation from the population value built into it.

That said, that large a discrepancy is unusual and might warrant further investigation (e.g., are the standard errors of the variance estimates very large).

Pat

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/8d178dbe-8d6d-4278-aa22-77de68f4fd43n%40googlegroups.com.

--

Patrick S. Malone, Ph.D., Malone Quantitative
NEW Service Models: http://malonequantitative.com

He/Him/His

[Janitor]

Hi Pat,

thanks for your quick answer, which already helps a lot. I still wonder, what your thought are on what to interpret / report in a paper. Usually, Journals (e.g., in  Psychology) ask for standardized coefficients, which, from a comparative/interpretative point of view makes sense. However, if I understood you correctly, the p-values of the standardized coefficients are not really representative of the data of the analyzed population. What do you recommend reporting in a paper, then?

Jan

[car...@web.de]

Hi, although I'm not Pat, here's my 2 cents. Always report the p values and confidence intervals for the unstandardized estimates and the standardized estimates as descriptive supplementary information. However, please do not create a figure with your results with the standardized coefficients and asterisks, which are then based on the unstandardized estimates. At least it confuses me every time and it happens often.

Am 17.03.21, 12:16 schrieb 'Janitor' via lavaan <lav...@googlegroups.com>:

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/fc542b36-edeb-44c0-b1b5-d7e67861eb75n%40googlegroups.com.

[Patrick (Malone Quantitative)]

I am Pat, and entirely concur. Standardized estimates are often useful (and the norm for factor loadings, in an exception), but unstandardized are the actual results of the model.

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/trinity-f0125a5c-b871-456b-be0b-e4415a9e128e-1615980670057%40msvc-mesg-web003.

[car...@web.de]

:-)

Am 17.03.21, 14:04 schrieb "Patrick (Malone Quantitative)" <mal...@malonequantitative.com>:

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/CAJc%3DyOEJ5N0ss%2BgnXFxbGwhX9ZbmL2d40rQY2pazWYfzodWT%3DQ%40mail.gmail.com.

[Janitor]

Thanks for the non-pat as well as the pat-answer :-) Thus, tables with unstandardized, standardized coefficients + unstandardized se's & p-vals, figures should either include standardized values without asterisks or p-values or unstandardized coeff. with asterisks or p-vals. Or did I understood something wrong?

[Rönkkö, Mikko]

If someone is interested in why the p value differs between different scaling approaches, this paper explains it quite nicely.

Gonzalez, R., & Griffin, D. (2001). Testing parameters in structural equation modeling: Every “one” matters. Psychological Methods, 6(3), 258–269. https://doi.org/doi:10.1037/1082-989X.6.3.258

If you want to get a p value that is not affected by latent variable scaling, you can fit a model were the focal parameter is constrained to be zero and do a likelihood ratio test between the constrained model and the original model.

Mikko

> To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/14b38765-7e69-4427-9695-812a486aa286n%40googlegroups.com.

[Denise Welsch]

Regarding reporting and p-values, get inspired by the app for automatic CFA with lavaan: https://statsomat.com 

It delivers an interpretation in plain English for your data. 

Dr. Denise Welsch

Rey Analytical Research 

https://reyar.de

https://statsomat.com 

Am 17.03.2021 um 14:45 schrieb 'Janitor' via lavaan <lav...@googlegroups.com>:

Thanks for the non-pat as well as the pat-answer :-) Thus, tables with unstandardized, standardized coefficients + unstandardized se's & p-vals, figures should either include standardized values without asterisks or p-values or unstandardized coeff. with asterisks or p-vals. Or did I understood something wrong?

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/14b38765-7e69-4427-9695-812a486aa286n%40googlegroups.com.