ANCOVA model for analysis of results from a pilot study

[Mohan Bairwa]

Dear friends at Medstats:

This is an analysis of pilot open-label RCT conducted to see the effect of yoga on quality of life in patients with chronic lung disease (change in the quality of life scores is the primary outcome and spirometry values (e.g. Forced Vital Capacity of Lung) are secondary outcomes). 

I am using ANCOVA model to see the effect of yoga (categorical variable, yoga + usual care vs usual care) on changes in quality of life post-intervention (continuous variable). Covariates will be included are - 1. oxygen supplementation - categorical; 2. baseline values of quality of life - continuous).

I will also test normality as well beforehand using some statistical tests including the Shapiro-Wilk test and box whisker plots, as there is a small sample size for this study, 25 patients in the intervention and control arm, each. Any comment on my approach would be appreciated, whether I am on the right path, or need some modification in my analysis approach.

[Bruce Weaver]

Hello Mohan.  Sorry about the delay in responding.  This forum has not been very busy of late, and as a result, I have not been checking as often as I used to! 

Let me start with the issue of doing a statistical test of normality.  IMO, this is a bad idea for several reasons.  I put together a short conference presentation on this issue several years ago.  You can view it here: 

https://www.researchgate.net/publication/299497976_Silly_or_Pointless_Things_People_Do_When_Analyzing_Data_1_Testing_for_Normality_as_a_Precursor_to_a_t-test

Second, take a look at this BMJ Stats Note by Bland & Altman (2009):

https://www.bmj.com/content/338/bmj.a3166

Finally, bear in mind that some authors have argued that inferential statistics have no place in a (true) pilot study.  E.g., see this resource and the articles it cites:

https://www.nccih.nih.gov/grants/pilot-studies-common-uses-and-misuses

HTH.