sample size for skewed data
Evie
I have done a sample size calculation for a continuous outcome between 2 groups (based on t-test). However the outcome will be heavily skewed so I would like to inflate the sample size to account for this. 2 questions: (1) do you think this is necessary? (2) have you any suggestions as to the inflation factor and a reference to back it up? I have currently used 15% as I've seen this used in a protocol but unable to find any firmer foundation. Thanks for your help.
Evan Kontopantelis
I have done a sample size calculation for a continuous outcome between 2 groups (based on t-test). However the outcome will be heavily skewed so I would like to inflate the sample size to account for this. 2 questions: (1) do you think this is necessary? (2) have you any suggestions as to the inflation factor and a reference to back it up? I have currently used 15% as I've seen this used in a protocol but unable to find any firmer foundation. Thanks for your help.
--
--
To post a new thread to MedStats, send email to MedS...@googlegroups.com .
MedStats' home page is http://groups.google.com/group/MedStats .
Rules: http://groups.google.com/group/MedStats/web/medstats-rules
---
You received this message because you are subscribed to the Google Groups "MedStats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to medstats+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
John Whittington
the basis of transformed data (transformed so as to be closer to Normal,
e.g. by log transformation if it's positively skewed)? If so, would not a
standard sample size estimation based on transformed data then be appropriate?
Kind Regards,
John
----------------------------------------------------------------
Dr John Whittington, Voice: +44 (0) 1296 730225
Mediscience Services Fax: +44 (0) 1296 738893
Twyford Manor, Twyford, E-mail: Joh...@mediscience.co.uk
Buckingham MK18 4EL, UK
----------------------------------------------------------------
Marc Schwartz
On Jun 5, 2014, at 8:51 AM, John Whittington <Joh...@mediscience.co.uk> wrote:
> At 06:38 05/06/2014 -0700, 'Evie' via MedStats wrote:
>> I have done a sample size calculation for a continuous outcome between 2 groups (based on t-test). However the outcome will be heavily skewed so I would like to inflate the sample size to account for this. 2 questions: (1) do you think this is necessary? (2) have you any suggestions as to the inflation factor and a reference to back it up? I have currently used 15% as I've seen this used in a protocol but unable to find any firmer foundation. Thanks for your help.
>
> Evie, if the data is very skewed, won't you be undertaking your analysis on the basis of transformed data (transformed so as to be closer to Normal, e.g. by log transformation if it's positively skewed)? If so, would not a standard sample size estimation based on transformed data then be appropriate?
>
> Kind Regards,
>
>
> John
In either case, it is not clear to me where the 15% inflation logic comes from, so I would recommend checking the references cited in the protocol you are basing this approach on to be sure that they could justify their use of it. It seems to me that if justified, the level of inflation would be correlated to, at minimum, how skewed the data are.
The t-test is relatively robust against non-normality, but you may give up effective power to reject the null by using it in that scenario. In that case, either transformation or the use of a non-parametric test may make sense.
Simulation would be easy to perform here, perhaps using a lognormal distribution to generate random samples for testing assumptions and running power/sample size assessments. This is easily done in R using various distribution generating functions.
Regards,
Marc Schwartz
MarkP
As others in this thread have said perhaps there is a tranformation that means you can analyse your data using a t-test.
Simulation is another approach, but it depends on your time & experience.
If you were going to use a non-parametric test instead, then the sample size would need inflating & here is a reference I have seen used.
This is a straightforward paper in the Journal: Optometry today, 2010 (July)
Prajapati, Bhavna; Dunne, Mark C.M. and Armstrong, Richard A. “Sample size estimation and statistical power analyses”
They use the context of the free software GPower3. On p4, Table 2 shows the asymptotic relative efficiency (ARE) of parametric/non-parametric tests.
There is a 95.5% efficiency between many of the common non-parametric tests, as compared to the parametric tests.
“The sample size required for a non-parametric test is
determined by multiplying the sample size calculated for an equivalent
parametric test by a correction factor. This correction factor is referred
to as the asymptotic relative efficiency (ARE) and as first described by
Pitman[13]”. Of course, AER is not the same as power. So the paper recommends an inflation factor of 1/0.955 of the t-test sample size for a study using the non-parametric equivalent (i.e. Mann-Whitney U).
Mark
John Whittington
If you were going to use a non-parametric test instead, then the sample size would need inflating & here is a reference I have seen used. .... They use the context of the free software GPower3. On p4, Table 2 shows the asymptotic relative efficiency (ARE) of parametric/non-parametric tests. ... There is a 95.5% efficiency between many of the common non-parametric tests, as compared to the parametric tests. .... "The sample size required for a non-parametric test is determined by multiplying the sample size calculated for an equivalent parametric test by a correction factor. This correction factor is referred to as the asymptotic relative efficiency (ARE) and as first described by Pitman[13]. Of course, AER is not the same as power. So the paper recommends an inflation factor of 1/0.955 of the t-test sample size for a study using the non-parametric equivalent (i.e. Mann-Whitney U).
Two points here ...
(1) the above surely relies on the fact that one is starting with a sample size calculation based on an appropriate 'equivalent parametric test'. If, for example, one (inappropriately) calculated a sample size for (untransformed) highly positively skewed data on the basis of a t-test, then one would very probably get an excessively high sample size estimate, because the SD of the untransformed data would be very high. The sample size for an 'appropriate' test (whether parametric or non-parametric) would then presumably result in a much smaller sample size estimate.
(2) even when the parametric test on which the initial sample size is based IS appropriate, the above correction factor is there to take into account the somewhhat lower power/ARE of the non-parametric test, regardless of the data distribution - i.e. not, as Evie, sugegsted, because the data were skewed.
That's how I see it, anyway.
Bruce Weaver
Abstract
The Wilcoxon-Mann-Whitney (WMW) test is often used to compare the means or medians of two independent, possibly nonnormal distributions. For this problem, the true significance level of the large sample approximate version of the WMW test is known to be sensitive to differences in the shapes of the distributions. Based on a wide ranging simulation study, our paper shows that the problem of lack of robustness of this test is more serious than is thought to be the case. In particular, small differences in variances and moderate degrees of skewness can produce large deviations from the nominal type I error rate. This is further exacerbated when the two distributions have different degrees of skewness. Other rank-based methods like the Fligner-Policello (FP) test and the Brunner-Munzel (BM) test perform similarly, although the BM test is generally better. By considering the WMW test as a two-sample T test on ranks, we explain the results by noting some undesirable properties of the rank transformation. In practice, the ranked samples should be examined and found to sufficiently satisfy reasonable symmetry and variance homogeneity before the test results are interpreted.
Copyright (c) 2009 John Wiley & Sons, Ltd.
Others have shown similar results--e.g., Zimmerman (2003) in another simulation study (https://www.researchgate.net/publication/233434621_A_Warning_About_the_Large-Sample_Wilcoxon-Mann-Whitney_Test).
HTH.
Bruce Weaver
http://imaging.mrc-cbu.cam.ac.uk/statswiki/FAQ/paranp
Evie
Thank you all VERY much for all your advice which was extremely useful. I appreciate you taking the time to answer.
Martin Holt
Founder of MedStats (a Google Group with over 1300 members, including just about all of the most notable Medical Statisticians)
Freelance Medical Statistician and Quality Expert
Contact me should you need the services of an experienced statistician
Persistence and Determination Alone are Omnipotent !
martin...@yahoo.com
4 Hawthorne Drive
Thornton
Coalville
LE67 1AW
Leicester
U.K.
Linked In
https://www.linkedin.com/profile/edit?trk=nav_responsive_sub_nav_edit_profile
Thank you all VERY much for all your advice which was extremely useful. I appreciate you taking the time to answer.
--