PLS Model Comparison
Ralf Plattfaut
Hi all,
I would like to compare two different models. A baseline (or small) model (with k_small predictors) and a large model that extends the small model with additional moderating variables (having k_large predictors). Both models have one dependent variable. Both models are tested using the same data set (with size N). Naturally, R² for the large model (R²_large) is greater than R² for the small one (R²_small). Moreover, adjusted R²_large is bigger than adjusted R²_small, too. How can I argue that this difference between R²_large and R²_small is statistically significant?
In OLS I could conduct an F-test to compare the two R²-values as described under http://psych.unl.edu/psycrs/statpage/rhtest_eg1.pdf:
F = ((R²_large – R²_small) / (k_large – k_small)) / ((1-R²_large)/(N-k_large-1))
Next, I could look at an F-table, e.g. F(1,N,alpha), and determine whether R²-change is significant at alpha-level.
Is this analysis valid for PLS, too?
Thank you very much. Best regards,
Ralf
Ned Kock
Hi Ralph.
The R-squared values from WarpPLS are calculated in exactly the same way as in multiple regression, for each LV block. The independent variables are the predictor LVs in each block, and the dependent variable is the criterion LV in each block. The LV scores are all standardized.
Another way in which you can compare models is by comparing path coefficients, not R-squared coefficients (see link below). Keep in mind that you (obviously) cannot compare models with different LVs using the F-statistic approach you linked, nor (I think) can you compare the same model with different samples.
This (i.e., path coefficient comparison) is a preferable approach, in my opinion, because it relies on coefficients calculated based on nonparametric techniques. This approach allows you to compare the same model with different samples. It also allows you to compare different models with the same LVs (although some removed), by using only the coefficients for the links being compared.
Finally, if you want to find out the absolute contributions of the predictor LVs to the R-squared on the corresponding criterion LV of an LV block, they are reported in WarpPLS as Cohen’s f-squared effect sizes.
Ned
Ralf Plattfaut
Hi Ned,
thank you very much for your answer. The clarification that R²-values from WarpPLS are calculated in exactly the same way as in multiple regression helps me a lot.
Based on this, I would argue that you can compare the R² of the different models as long as one smaller model is a subset of the larger model using the F-statistic approach. In his dissertation Richards used the same approach to compare a baseline model and an extended model including three new fit variables (http://www.bauer.uh.edu/doctoral/documents/RichardsDissertationFinal.doc). He used PLSGraph for his analysis.
However, I will also take a look at the Cohen’s f-squared effect sizes.
Once again, thank you very much.
Best regards,
Ralf
--
You received this message because you are subscribed to the Google
Groups "PLS-SEM" group.
To post to this group, send email to pls...@googlegroups.com
To unsubscribe from this group, send email to
pls-sem+u...@googlegroups.com
Ned Kock
I’d like to add a qualification to my comment that: “The R-squared values from WarpPLS are calculated in exactly the same way as in multiple regression …”
This comment applies to linear analyses. If one of the nonlinear algorithms is used in WarpPLS, the corresponding R-squared coefficients will be different.
Typically, but not always, nonlinear R-squared coefficients will be slightly higher than linear coefficients.
Their interpretation, however, is the usual one – they are the percentages of explained variance in the criterion (endogenous) LVs by the predictor LVs.