Hypothesis acceptance or rejection

[Ahamed AFM Jalal]

Respected Group members,

 

Thank you very much for all your kind supports. I am dealing with a problem, which I am stating below and seeking your kind suggestion:

 

I have two hypotheses,

H1: X is negatively related to Y.

H2: X is negatively related to Z.

 

I run the model in SmartPLS, and have the following results

H#	Relationship	Path Coefficient	T-statistic	P value	Statistical Sign.	Hypothesis support
H1	X-> Y	0.13	3.63	0.0004	Sign	No
H2	X-> Z	-0.09	2.34	0.0205	Sign	Yes

Note: two-tailed test

 

As the table shows, both the hypotheses are statistically significant. The sign of hypothesis 1 is positive , which I conceptualize a negative relationship, so I said that my data and analysis did not support the hypothesis. For, Hypothesis 2 , the relationship is conceptualize negative and the sign of the path coefficient is negative, so I accept this hypothesis.

 

I had submitted this paper to a journal. The honourable reviewer comment as:

 

“You say that you are using the bootstrap procedure but Table 4 does not show very stable path coefficients. For instance, to be significant path coefficients are supposed to be greater than 0.2. Why did you support H2 and not supported H1? I did not understand the rationale behind the acceptance and rejection of hypotheses.”

 

Could you please help me in this regard? Thanks in advance

 

Jalal

---

[Ned Kock]

Hi Jalal.

 

I think that path coefficients you listed are too low to be seen as supporting the hypothesized effects without an inspection of effect size measures, even with the low P values.

 

In WarpPLS, Cohen’s f-squared effect size coefficients are calculated and shown for all path coefficients. These are calculated as the absolute values of the individual contributions of the corresponding predictor latent variables to the R-square coefficients of the criterion latent variable in each latent variable block.

 

With these effect sizes users can ascertain whether the effects indicated by path coefficients are small, medium, or large. The values usually recommended are 0.02, 0.15, and 0.35; respectively (Cohen, 1988). Values below 0.02 suggest effects that are too weak to be considered relevant from a practical point of view, even when the corresponding P values are statistically significant; a situation that may occur with large sample sizes.

 

By the way, P values are very sensitive to sample size. With a large enough sample, anything will be significant; even minute “effects” due to error.

 

See the WarpPLS User Manual for the reference cited and other references.

 

Ned 

http://nedkock.com

[Ahamed AFM Jalal]

Thank you very much Professor Ned for your kind response. The sample size is 166.

 

Thanks Wissem, for the question you asked: " I'm trying to find a way to calculate the effect size for each path coefficient using smartPLS. Is there a way to do it?" If you have any good idea please let me know.

Best regards

Jalal

---

From: pls...@googlegroups.com [pls...@googlegroups.com] on behalf of Ned Kock [ned...@scriptwarp.com]
Sent: 08 March 2013 16:36
To: pls...@googlegroups.com
Subject: [pls-sem] You need effect sizes for hypothesis acceptance or rejection when effects appear to be small (e.g., <.15)

--
--
You received this message because you are subscribed to the Google
Groups "PLS-SEM" group.
To post to this group, send email to pls...@googlegroups.com
To unsubscribe from this group, send email to
pls-sem+u...@googlegroups.com
 
---
You received this message because you are subscribed to the Google Groups "PLS-SEM" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pls-sem+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[BUHMANN Alexander]

Hi Ned,

I totally agree with you that the these coefficients are not a good basis for support of the hypotheses. Im writing, because I have trouble following your argument. May be I am missing the point, but – according to your rationale – shouldn't both of Jalal's path coefficients still be okay? They are indeed very small, but would – when you take 0.02 as a cut of value – still be worth considering, right?

Thanks allot for your comments!

Alex

[Ned Kock]

Hi Alex. My guess is that, with those path coefficients, the corresponding f-square coefficients may well be below 0.02. But without actually calculating the effect sizes, one can only guess.

[BUHMANN Alexander]

Okay, I see! Thanks Ned!

Am 08.03.2013 um 22:34 schrieb Ned Kock:

Hi Alex. My guess is that, with those path coefficients, the corresponding f-square coefficients may well be below 0.02. But without actually calculating the effect sizes, one can only guess.

[Johnny Amora]

Hi everyone,

We also consider that the smaller the effect size, the larger the sample size needed. I think it would be nice to consider that one may conduct power analysis first to determine the sample size before the SEM or Regression analysis using SmartPLS or WarpPLS is done.

Johnny

--

Johnny Amora
Statistician, De La Salle-College of Saint Benilde
Taft Avenue, Manila

Mobile: 09289301119

[Ramayah T]

In SmartPLS you can also calculate the effect size for each path by using the formula where you calculate the f squared using the R squared included and R squared excluded for each path. Using this formula you can check for the individual path effect sizes.

See attachment.

Regards,

T. Ramayah

--

--
You received this message because you are subscribed to the Google
Groups "PLS-SEM" group.
To post to this group, send email to pls...@googlegroups.com
To unsubscribe from this group, send email to
pls-sem+u...@googlegroups.com
 
---
You received this message because you are subscribed to the Google Groups "PLS-SEM" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pls-sem+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
http://www.ramayah.com/

[Christian Ringle]

Thanks T. Ramayah!

Building on your explanations, I posted� the "How Tos" for the f� effect size computation in SmartPLS on www.facebook.com/smartpls. You also find an example in that post.

Best
Christian
www.tuhh.de/hrmo

Am 09.03.2013 03:35, schrieb Ramayah T:

In SmartPLS you can also calculate the effect size for each path by using the formula where you calculate the f squared using the R squared included and R squared excluded for each path. Using this formula you can check for the individual path effect sizes.

See attachment.

Regards,

T. Ramayah

On Fri, Mar 8, 2013 at 11:36 PM, Ned Kock <ned...@scriptwarp.com> wrote:

Hi Jalal.

�

I think that path coefficients you listed are too low to be seen as supporting the hypothesized effects without an inspection of effect size measures, even with the low P values.

�

In WarpPLS, Cohen�s f-squared effect size coefficients are calculated and shown for all path coefficients. These are calculated as the absolute values of the individual contributions of the corresponding predictor latent variables to the R-square coefficients of the criterion latent variable in each latent variable block.

�

With these effect sizes users can ascertain whether the effects indicated by path coefficients are small, medium, or large. The values usually recommended are 0.02, 0.15, and 0.35; respectively (Cohen, 1988). Values below 0.02 suggest effects that are too weak to be considered relevant from a practical point of view, even when the corresponding P values are statistically significant; a situation that may occur with large sample sizes.

�

By the way, P values are very sensitive to sample size. With a large enough sample, anything will be significant; even minute �effects� due to error.

�

See the WarpPLS User Manual for the reference cited and other references.

�

Ned�

http://nedkock.com

--
--
You received this message because you are subscribed to the Google
Groups "PLS-SEM" group.
To post to this group, send email to pls...@googlegroups.com
To unsubscribe from this group, send email to
pls-sem+u...@googlegroups.com

�

---
You received this message because you are subscribed to the Google Groups "PLS-SEM" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pls-sem+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

�
�

--
http://www.ramayah.com/ --

--
You received this message because you are subscribed to the Google
Groups "PLS-SEM" group.
To post to this group, send email to pls...@googlegroups.com
To unsubscribe from this group, send email to
pls-sem+u...@googlegroups.com

�

---
You received this message because you are subscribed to the Google Groups "PLS-SEM" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pls-sem+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

�
�

[Ned Kock]

Hi Jalal.

 

Looking at your path coefficients and P values, and considering the N=166, I am a bit curious.

 

What was the resampling algorithm (e.g., bootstrapping) that you used to get those P values?

 

I ask because I don’t usually see P values that low, especially with a two-tailed test, for path coefficients that low at the N you mentioned.

 

Those combinations of low path coefficients and low P values are not very common even at N’s around 200.

 

Ned

 

 

 

From: pls...@googlegroups.com [mailto:pls...@googlegroups.com] On Behalf Of Ahamed AFM Jalal
Sent: Friday, March 08, 2013 6:51 AM
To: pls...@googlegroups.com
Subject: [pls-sem] Hypothesis acceptance or rejection

[…]

[Ahamed AFM Jalal]

Dear Professor Ned and Honorable group members,

 

Thank you very much for your very insightful discussions. 

In particular, I am answering the last mail of Prof. Ned:

 

Regarding the bootstrapping, if I understood you correctly, I used the followings

Sign changes: No sign change

Cases: 166

Samples: 500

To get the P values I used http://graphpad.com/quickcalcs/PValue1.cfm

 

For your kind information, I started redoing & rechecking all my calculation once again, just in case if I did any mistake.

 

Thank you

 

Regards

Jalal

---

From: pls...@googlegroups.com [pls...@googlegroups.com] on behalf of Ned Kock [ned...@scriptwarp.com]

Sent: 09 March 2013 19:26
To: pls...@googlegroups.com
Subject: RE: [pls-sem] Hypothesis acceptance or rejection

--

--
You received this message because you are subscribed to the Google
Groups "PLS-SEM" group.
To post to this group, send email to pls...@googlegroups.com
To unsubscribe from this group, send email to
pls-sem+u...@googlegroups.com