Error in JASP inherited from lavaan

[Mikelavaan]

I'm using mediation analysis in JASP, which uses lavaan. And have the following problem, which I've posted in this JASP forum:

https://forum.cogsci.nl/discussion/comment/24031#Comment_24031

And people are saying, by interpreting the error message returned, that the error isn't in JASP, but in the underlying lavaan. 

The problem is:

When I do mediation analysis with one of the columns (for the Mediator variable) having numbers around as small as the following:

0.000798458

I get an error and no result returned. When I multiply all the numbers by 10,000 to create a new column, where all numbers are greater than 1, and I use this column instead, it all works ok. So, this is a workaround. But can it really be the case that everything gets thrown by small numbers (which aren't really that small)?

A different workaround, which works, is to tick the “Standardized estimates” box.

But are my workarounds valid? Are the results I get with this workaround(s) trustworthy?

The better thing would be that JASP mediation analysis could just work with my raw numbers. Do you know why it cannot? Is my estimation that it is because the numbers are too small correct?

Actually, the whole column is thus:

0.000798458

0.000771

0.001164574

0.001319925

0.001804233

0.002713422

0.002418353

0.004975053

0.003740752

0.005934723

0.006377

0.010447035

[Yves Rosseel]

Dear Michael,

This sounds like a bad-scaling problem. In the world of SEM, you need to
make sure that the scales of all the variables are more or less equal to
each other, and (ideally) roughly equal to one.

You may think that 0.000798458 is not small, but when we compute
variances/covariances/hessian/gradients, these values are multiplied
with each other, leading to even smaller numbers. For example, the
square is 0.0000006375352. Computations with very small numbers may lead
to numerical problems, known as 'numerical underflow'.

So rescaling this variable is a good idea.

Yves.

[Mikelavaan]

Thank you. 

So, you are saying that my work-around (multiplying the column of small numbers by 10,000) isn't some crude, ugly work-around, but is in fact what should be done? It is best practice? And that the results of such mediation analysis are trustworthy?  

On a more advanced point (just for my interest) - could this be incorporated into the lavaan algorithm, to do such a multiplication when a column of small numbers is entered, such that it all works, without the user even knowing that this has been done? If this cannot be done at the lavaan level, perhaps it could be done at the JASP level, for cases where JASP is the entry point to lavaan?  

Thank you again Yves for your help and guidance, and lavaan in the first place.

P.S. you wrote: "In the world of SEM, you need to make sure that the scales of all the variables are more or less equal to each other, and (ideally) roughly equal to one". Is this what selecting the option of “Standardized estimates” in JASP, to standardize (mean = 0, standard deviation = 1) all the variables before estimation, does? 

[Pat Malone]

Re-scaling your input variables for SEM is both good and common practice. It will not affect your test statistics. Just remember to back-transform your resulting coefficient estimates.

Re the PS, standardizing your input variables would do this, but it has disadvantages--in particular, that you are now dividing by a quantity (the SD) that is estimated with error, instead of a constant which is exactly known. Having all the observed variances be identical also requires special handling. Unless you care about factor means (e.g., in a trajectory model), setting observed means to zero has no particular effect.

Pat

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/5770bab0-6725-4742-8d0c-f353c0e254f6n%40googlegroups.com.

--

Patrick S. Malone, PhD

Sr Research Statistician, FAR HARBΦR

+1 803.553.4181  |  pat@  |  farharbor.com

This message may contain confidential information; if you are not the intended recipient please notify the sender and delete the message.

[Mikelavaan]

Thank you to everyone. 

If I could please trouble on another point 

        - do you know a good method to get a p-value from a Confidence Interval (CI) from a Percentile bootstrap? 

The p-value that JASP reports when using Percentile bootstrap doesn't actually derive from the CI (which even means the p-value and CI can contradict in some cases).

[Pat Malone]

There might be a way, but I wouldn't recommend doing so. The CI contains more information than the "estimate divided by SE" that gives the conventional p-value. Also the CI may be asymmetric (which is often the point of the percentile bootstrap), in which case it would definitely not be expected to map onto the JASP result. Just report the CI, and if the reporting structure necessitates comparison to a null hypothesis, you can observe whether the CI excludes zero, or whatever..

Note that the result of whether it excludes zero will match the est/SE significance test if all assumptions going into the normal-theory SE are met (including symmetry of the sampling distribution) and you have infinite bootstrap replications. If the SE assumptions are not met, the bootstrap CI result will be more trustworthy (deliberately not going into the different flavors of bootstrap here).

Pat

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/2eb13948-f12a-42c2-bb09-84aa8752bfdfn%40googlegroups.com.

[car...@web.de]

Technically speaking, this is not a problem. All you have to do is find the CI that no longer contains the zero. Whether this can be interpreted meaningfully can certainly be seen in one way or another.

Am 24.08.21, 20:02 schrieb 'Pat Malone' via lavaan <lav...@googlegroups.com>:

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/CAJhmz4cGSAHjPcjjNeXsXz0whkn_FvGCZTXjK9eqDAxoh2QknQ%40mail.gmail.com.

[Mikelavaan]

Thank you. I understand what you mean about interpreting the CI. 

It is just that this work is for a biology audience, and so they are used to seeing p-values. And absence thereof will probably have them assuming some skulduggery (!). 

JASP does give a p-value, but not derived from the bootstrap CI. 

But if you guys don't know a way to get a p-value from the CI, then I am happy to take that it just isn't possible (especially heeding the asymmetry problem highlighted). And so the audience will have to make do with the reported CI, and pointing out that zero is not contained therein. 

I have found this paper, which teaches how to get a p-value from a CI: https://www.bmj.com/content/343/bmj.d2304 

BUT I assume that this method isn't applicable in this Percentile bootstrap CI case? 

[Pat Malone]

Again, JASP's p-value is based on different theory than the bootstrap, and requires more specific assumptions. Do not expect them to match.

But Carl (carnold), gave you a way, albeit briefly stated, to obtain the p-value. Check the tails of the bootstrapped distribution to find out which level of CI (96.3, 97.9, etc.) puts the CI adjacent to the zero border, then subtract from 100. Though be aware that the smaller the part of the tail you are cutting off, the more bootstrap samples you'll want to have to stabilize the confidence limits.

Pat

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/fb6e30de-edf3-49ee-a7d3-daec8cea3b31n%40googlegroups.com.