Comparing R squared values
Geoff Soutar
Hi
Just wondered if anyone new a good way to compare R squared values between two competing models. I know you can use Steiger’s Z in regression, but you need to output estimated values for the dependent variable in the two models to do this, as you need to find the correlation between these two sets of scores and I cannot see how this might be done in an SEM unless you did the calculation yourself in excel or some other program. Wondered if there was a way to get such output directly.
Thanks a lot
Geoff Soutar
Ned Kock
Hi Geoff.
I’ve done some simulations to see if I could answer your question using the nonparametric, distribution-free, approach for estimation of P values employed by WarpPLS. I am assuming LV blocks with several Xs pointing at one Y in each block. We would consider each block as being a model, for the purposes of this discussion.
I am also assuming that the correlations among the Xs are relatively low, which would normally be the case if the Xs are LVs and their full collinearity VIFs are below the 3.3 threshold. The full collinearity VIFs are reported by WarpPLS.
It seems that you will get a fairly good approximation if you use the Satterthwaite procedure for multi-group analysis discussed in the following post on the WarpPLS Blog. But with one extra step: you will have to estimate the standard errors of the R2s.
On the Excel sheet, you’ll enter the sample sizes for the two models, the R2s (instead of the betas), and the standard errors for the R2s. The latter will have to be estimated based on the standard errors of the betas for the links between the Xs and Y, which are reported by WarpPLS.
The standard error for the R2 of each model {SER2} will be calculated as the mean standard error for each of the links between the Xs and Y {Mean(SEi)} multiplied by the square root of: the number of Xs divided by 2 {(p/2)^.5}.
SER2 = Mean(SEi) * (p/2)^.5
The outputs will be P values in the Excel spreadsheet; use the one-tailed value for the Satterthwaite method.
As you’ll see, the greater the number of Xs (p, in the formula), the greater will be the estimated standard error for each R2. Thus, the greater the p, the larger the difference between R2s will have to be for it to be significant.
Sorry, I can’t see a more direct approach without making parametric assumptions. The software could do this comparison directly using the actual SER2s, but currently WarpPLS doesn’t do that.
Ned
Geoff Soutar
Hi Ned
Thanks a lot – it was the SER2 that was the problem – this solves that
Geoff
--
--
You received this message because you are subscribed to the Google
Groups "PLS-SEM" group.
To post to this group, send email to pls...@googlegroups.com
To unsubscribe from this group, send email to
pls-sem+u...@googlegroups.com
---
You received this message because you are subscribed to the Google Groups "PLS-SEM" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pls-sem+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.