Hypothesis testing with proportions
bron...@net.hr
Hi there,
i'm not sure if it's legitimate to use the t-test with the following problem:
I'm analysing the proportions of mutant and wild-type DNA.
In every sample the proportion of mutant DNA is measured.
Making 45 samples we recorded a mean proportion of mutant of 29.1% with a standard deviation of 3.5%.
I want to test if this proportion is significantly different from 50% mutant and 50% wild-type.
Usually I would use the t-test. But the distribution here can't be the t-distribution, because it is restricted from 0 to 100%.
What kind of test am I supposed to do?
Your help is highly appreciated.
Bronzing
Richard Ulrich
Given a mean of 29% and close clustering around it,
it is probably pretty convincing to point out to folks that
(say) 44 of 45 are less than 50% -- a sign test.
If that 3.5% is not the SD but is the SE of the mean, then
you have wider spread of scores, but the same reasoning holds.
But I don't think any statisticians are going to fault you for
using a one-sample t-test to compare 29% to 50%.
Most variables that we test do not have infinite ranges.
Sometimes we want to transform percentages that are close
to zero, to achieve better modeling near zero, but your
concern is "50" so that transforming is not likely to gain
much, if anything.
--
Rich Ulrich, wpi...@pitt.edu
http://www.pitt.edu/~wpilib/index.html
Alan J. Salmoni
Er, aren't the figures you quote percentages and not proportions?
<grin> - similar things (I think - anyone correct me if I'm wrong -
that you should take the percentage and divide by 100 so you are
comparing 0.291 for mutant and 0.709 for wildtype against 0.5 mutant
and wildtype both).
I think a t-test is okay for these data. The main assumptions are that
the data are continuous (which I think these data are - Nunally
mentioned that nonparametric scales of 11 or more points could be
considered continuous), and that each condition produces normal
distributions of equal variances (homogeneity of variance). The limits
of the distribution shouldn't matter here, but you must make sure of
the homogeneity. Apart from that, you should be okay. I know of people
who have analysed both proportions and percentages using ANOVA and
t-tests and not been questioned at conferences. Having said that, I've
seen simple Likert-type scales analysed with t-tests get past peer
review..!
Again, anybody please correct me if I have made misconceptions.
Alan.
Reef Fish
sci.stat.math,
in which the procedure of testing proportions has been used as an
example for computing p-value. The reference given there,
RF> See e.g., http://tinyurl.com/8hrw8
should show the OP how to do a test of PROPORTION p, with Z.
Richard Ulrich wrote:
> On Sun, 26 Jul 2005 9:54:44 GMT, bron...@net.hr wrote:
>
> >
> > Hi there,
> >
> > i'm not sure if it's legitimate to use the t-test with the following problem:
No, it's not.
> > I'm analysing the proportions of mutant and wild-type DNA.
> > In every sample the proportion of mutant DNA is measured.
> > Making 45 samples we recorded a mean proportion of mutant of 29.1% with a standard deviation of 3.5%.
Since you're treating these samples as samples from the SAME
population, all you have to do is to combine the proportion data
into ONE single estimate of the unknown proportion p,
p-hat = (total of mutant DNAs in all 45 samples)/(Total sample size).
This p-hat is the APPROPRIATE estimate and test statistic of what
the OP was trying to test by the (inappropriate) mean of the
proportions from 45 samples.
> > I want to test if this proportion is significantly different from 50% mutant and 50% wild-type.
> > Usually I would use the t-test. But the distribution here can't be the t-distribution, because it is restricted from 0 to 100%.
> > What kind of test am I supposed to do?
It's a straightforward problem in testing a single proportion p.
Ho: p = 0.5 See solution in http://tinyurl.com/8hrw8
Now comes with Richard Ulrich with his characteristic
ERRORS, bad advice, and statistical QUACKERY:
> Given a mean of 29% and close clustering around it,
> it is probably pretty convincing to point out to folks that
> (say) 44 of 45 are less than 50% -- a sign test.
>
> If that 3.5% is not the SD but is the SE of the mean, then
> you have wider spread of scores, but the same reasoning holds.
>
> But I don't think any statisticians are going to fault you for
> using a one-sample t-test to compare 29% to 50%.
ANY statistician or non-statistician who knows anything about
testing a single proportion is going to fault the OP and Ulrich
for using a t-rest on 45 samples of proportions instead of
doing a straightforward test based on a SINGLE proportion
estimate, as detailed in the web link.
> Most variables that we test do not have infinite ranges.
> Sometimes we want to transform percentages that are close
> to zero, to achieve better modeling near zero, but your
> concern is "50" so that transforming is not likely to gain
> much, if anything.
>
> --
> Rich Ulrich, wpi...@pitt.edu
> http://www.pitt.edu/~wpilib/index.html
Irrelevant NONSENSE for the OP's problem in question. A
T-test is a completely inappropriate test for p = 0.5.
-- Bob.
Richard Ulrich
<Large_Nass...@Yahoo.com> wrote:
> This topic is within the ongoing discussion of p-values in
> sci.stat.math,
> in which the procedure of testing proportions has been used as an
> example for computing p-value. The reference given there,
>
> RF> See e.g., http://tinyurl.com/8hrw8
>
> should show the OP how to do a test of PROPORTION p, with Z.
That link has a nice note, for what it does.
The reader might worry that the examples there each have
a 0/1 criterion. The note does not deal with a collection
of 45 proportions that are based on individual Ns, and which
might *not* be homogeneous.
The note also contains a detailed description of a power
analysis, which is a subject that Bob previously found
to be baffling and infuriating "gibberish". Well, he has said
that six or eight times about my terse overview -- he never
has cited the several lines of actual description that followed.
>
> Richard Ulrich wrote:
> > On Sun, 26 Jul 2005 9:54:44 GMT, bron...@net.hr wrote:
> >
> > >
> > > Hi there,
> > >
> > > i'm not sure if it's legitimate to use the t-test with the following problem:
>
> No, it's not.
- not a quoted word, so far, has been mine. The t-test is robust
against heterogeneity of the set of proportions and against disparate
Ns, which Bob's solution is not.
The sign test, which is quoted down below, is even more robust.
>
>
> > > I'm analysing the proportions of mutant and wild-type DNA.
> > > In every sample the proportion of mutant DNA is measured.
> > > Making 45 samples we recorded a mean proportion of mutant of 29.1% with a standard deviation of 3.5%.
>
> Since you're treating these samples as samples from the SAME
> population, all you have to do is to combine the proportion data
> into ONE single estimate of the unknown proportion p,
>
> p-hat = (total of mutant DNAs in all 45 samples)/(Total sample size).
>
> This p-hat is the APPROPRIATE estimate and test statistic of what
> the OP was trying to test by the (inappropriate) mean of the
> proportions from 45 samples.
Bob is right, to an extent -- testing the overall proportion
is the fine test, if you are willing to make the assumption
of homogeneity, and if the Ns in each sample are similar.
The OP should probably test the homogeneity, if going
that route.
>
> > > I want to test if this proportion is significantly different from 50% mutant and 50% wild-type.
> > > Usually I would use the t-test. But the distribution here can't be the t-distribution, because it is restricted from 0 to 100%.
> > > What kind of test am I supposed to do?
RF >
> It's a straightforward problem in testing a single proportion p.
>
> Ho: p = 0.5 See solution in http://tinyurl.com/8hrw8
- same reference already cited -
RF >
> Now comes with Richard Ulrich with his characteristic
> ERRORS, bad advice, and statistical QUACKERY:
My advice is based on 30 years of experience, doing research
and giving advice to researchers. Bob Ling's experience,
as much he has documented so far, has been in academia,
teaching classes. There have been a number of examples where
his inexperience in relevant research seems to shine through.
Bob's test of p is the ivory-tower solution, with problems that
I named. Most reviewers and readers will rightfully want more
details since they are readily available.
RU >
> > Given a mean of 29% and close clustering around it,
> > it is probably pretty convincing to point out to folks that
> > (say) 44 of 45 are less than 50% -- a sign test.
> >
> > If that 3.5% is not the SD but is the SE of the mean, then
> > you have wider spread of scores, but the same reasoning holds.
> >
> > But I don't think any statisticians are going to fault you for
> > using a one-sample t-test to compare 29% to 50%.
RF >
> ANY statistician or non-statistician who knows anything about
> testing a single proportion is going to fault the OP and Ulrich
> for using a t-rest on 45 samples of proportions instead of
> doing a straightforward test based on a SINGLE proportion
> estimate, as detailed in the web link.
The link had examples with 0/1 for each sampling.... On the
face of it, simpler. Not the same. It might work for data like
these, but it leaves questions. (For the data as described, it
looks like *anything* would work.)
>
RU >
> > Most variables that we test do not have infinite ranges.
> > Sometimes we want to transform percentages that are close
> > to zero, to achieve better modeling near zero, but your
> > concern is "50" so that transforming is not likely to gain
> > much, if anything.
[ snip, tag]
- more good advice.
RF >
> Irrelevant NONSENSE for the OP's problem in question. A
> T-test is a completely inappropriate test for p = 0.5.
Probably irrelevant for the OP; far from nonsense.
Bob might look up "one-sample t-test," if that is his problem.
m00es
arcsin transformation) is often applied before analyzing proportions in
such a manner. Therefore, take each proportion and apply
arcsin( sqrt(p) )
which removes, approximately, the dependency of the variance on the
mean.
Best,
m00es
bron...@net.hr
Thanks so far for the help.
Just to clarify my problem (coz I'm not sure if you got me right and the proposed
procedures are applicable):
We made 45 gels from tissue samples of one patient which contain partially mutant
partially wild-type DNA. From the IODs (integrated optical desities) of the bands on
the gels we calculated for each of the 45 gels the proportion of mutant DNA
contained in the gel. The mean value of all the 45 proportions is 0.291 with a
standard deviation of 0.035.
The proposed web-site deals with samples taken from populations, each sample
producing an integer number. (Am I wrong?) I don't know how this is comparible to
my problem.
I posted the same question to the forim of OriginLab (the software I'm using for my
calculations). Here's the answer I got until now. What do you think?
> minimax
> China
> Posted - 07/26/2005 : 9:46:36 PM
> --------------------------------------------------------------------------------
> Hi bronzing,
> You maybe have a misunderstanding on (One-Sample) t-test. T-test on a
> variable does not mean that the variable should obey t-distribution. It indeed
> requires the variable obey normal distribution. The t-test gives a
> Student's t statistic t=sqrt(n)*(mean(y)-u)/sd(y). Assuming that the null
> hypothesis (H0: mean = u) is true and the population is normally distributed,
> the t statistic has a Student's t distribution with n-1 degrees of freedom.
> Therefore, your case require that the proportion obeys normal ditribution, not
> t-distribution. Hence, you can first test whether the proportion is normally
> distributed using menu Statistics-Normality test(Shapiro-wilk). Once the
> proportion passes normality test, you can use t-test; otherwise,
> nonparametric tests, which will be integrated in Origin version8, such as the
> sign rank test should be used.
>
> Moreover, note that the one-sample t-test is appropriate in this situation
> because the standard deviation of the population from which the data arise
> is unknown. When you know the standard deviation of the population, use
> the One-Sample Z-Test.
bron...@net.hr
Am 27.07.2005 um 11:21 wurde geantwortet auf:
"m00es" <m0...@yahoo.com> schrieb:
Sorry I don't get it. What does this function do? Am I supposed to apply this
function to each proportion and then do the t-test with the new set of numbers
with H0: arcsin (sqrt(0.5))?
bron...@net.hr
> Hi Bronzing,
>
> Er, aren't the figures you quote percentages and not proportions?
> <grin> - similar things (I think - anyone correct me if I'm wrong -
> that you should take the percentage and divide by 100 so you are
> comparing 0.291 for mutant and 0.709 for wildtype against 0.5 mutant
> and wildtype both).
Yea, you'r right. Sorry for the mess-up.
> I think a t-test is okay for these data. The main assumptions are that
> the data are continuous (which I think these data are - Nunally
> mentioned that nonparametric scales of 11 or more points could be
> considered continuous), and that each condition produces normal
> distributions of equal variances (homogeneity of variance). The limits
> of the distribution shouldn't matter here, but you must make sure of
> the homogeneity. Apart from that, you should be okay. I know of people
> who have analysed both proportions and percentages using ANOVA and
> t-tests and not been questioned at conferences. Having said that, I've
> seen simple Likert-type scales analysed with t-tests get past peer
> review..!
In the OriginLab-forum "minimax" suggests testing the population for normal
distribution first and then applying the t-test (see my reply in the Richard Ulrich/
Reff Fish thread). What do you think?
bye
Bronzing
Bruce Weaver
----- snip ------------
Here's a question for bronzing: Why did you collect 45 samples instead
of one large sample?
--
Bruce Weaver
bwe...@lakeheadu.ca
www.angelfire.com/wv/bwhomedir
m00es
variance stabilizing transformations.
The variance of a proportion depends on the proportion itself.
Therefore, if the proportions in the 45 gels differ, you get
heterogeneous variances. That violates the homogeneity of variance
assumption underlying the t-test. The arcsin transformation removes
(approximately) that dependency.
A note about the suggestion from the OriginLab forum. Testing for
normality is unnecessary. Your sample does NOT have to come from a
normal distribution if you want to apply the t-test. The *sampling
distribution of the mean* (in your case, the mean of the 45
proportions) needs to be normally distributed. Based on the central
limit theorem, you can be pretty darn certain that the mean of your 45
proportions is close to normal, even if the the individual proportions
are not normally distrbuted.
m00es
Bruce Weaver
> Yes, correct. The arcsin transformation is one of many so-called
> variance stabilizing transformations.
>
> The variance of a proportion depends on the proportion itself.
I'm with you to here.
> Therefore, if the proportions in the 45 gels differ, you get
> heterogeneous variances. That violates the homogeneity of variance
> assumption underlying the t-test. The arcsin transformation removes
> (approximately) that dependency.
---- snip -----
Now you've lost me. We're talking about a single-sample t-test here,
right? The data are 45 proportions. The SE for the proposed
single-sample t-test is the SD of the 45 proportions divided by the
square root of 45. Where do you see the variances of the individual
proportions themselves coming into play and violating the assumptions
of the test?
To the OP: You've had at least 3 suggestions:
1) single-sample t-test on the 45 proportions
2) combine all samples and do a z-test on the one big sample
3) do an arcsine transformation of the 45 proportions, and do the
single-sample t-test on that.
According to your original post, all you want to know is whether the
observed proportion of mutant DNA is different from 0.5. I think you'll
find ALL of those methods (plus any others that might be suggested)
generate p-values that are well below 0.05 (e.g., for option 1 above, t
= -40.06, p < 0.001). So in this case, it's probably not worth
splitting hairs about how many angels can dance on the head of a pin.
(How's that for a mixed metaphor?) I'd be more concerned about choice
of test if the p-values were borderline, and different approaches were
leading to different decisions.
Thom
news:1122410561....@g49g2000cwa.googlegroups.com...
> t-tests and not been questioned at conferences. Having said that, I've
> seen simple Likert-type scales analysed with t-tests get past peer
> review..!
... and why not. If the residuals are approximately normal, the variances
approximately equal (for the two sample test) and the independence
assumption tenable the t test will give you pretty sensible results for a
Likert scale, proportions etc. In these cases the alternatives
(transformation etc.) should give you very similar results. the main reason
to prefer the t test (or a CI based on t) is probably for clarity of
presentation.
Thom
Reef Fish
Bruce Weaver wrote:
> Richard Ulrich wrote:
> > On 26 Jul 2005 19:58:45 -0700, "Reef Fish"
> > <Large_Nass...@Yahoo.com> wrote:
> >
> > > This topic is within the ongoing discussion of p-values in
> > > sci.stat.math,
> > > in which the procedure of testing proportions has been used as an
> > > example for computing p-value. The reference given there,
> > >
> > > RF> See e.g., http://tinyurl.com/8hrw8
> > >
> > > should show the OP how to do a test of PROPORTION p, with Z.
> >
> > That link has a nice note, for what it does.
> > The reader might worry that the examples there each have
RU > a 0/1 criterion. The note does not deal with a collection
RU > of 45 proportions that are based on individual Ns, and which
RU > might *not* be homogeneous.
This is the kind of nonsense of which Richard Ulrich is infamous.
If p(i)-hat= k(i)/N(i), i = 1, ..., 45, everyone SHOULD know the
combined proportion is NOT Sum<i=1,...45> (p(i)-hat)/45, for
precisely the reason that the N(i)s may be considerably different.
Ulrich's use of the word "homogeneous" is wrong on two counts --
(a) wrong word for different sample sizes; (b) for his reason.
That's why the combined proportion estimate is
p-hat = (sum<i> k(i))/(sum<i> N(i)).
bronzing > i'm not sure if it's legitimate to use the t-test with the
following problem:
RF > No, it's not.
RU > - not a quoted word, so far, has been mine. The t-test is robust
RU > against heterogeneity of the set of proportions and against
disparate
RU > Ns, which Bob's solution is not.
Only in THIS statistics newsgroup could one find a poster (Richard
Ulrich) who claims to be statistician, made a clear blunder in
advising the OP of using the WRONG procedure for a well-known
standard method of dealing with proportions, but has the GALL to
come back to defend his Quackery!!!
RU > The sign test, which is quoted down below, is even more robust.
Quackery for bronzing's problem.
bronzing > I'm analysing the proportions of mutant and wild-type DNA.
bronzing > In every sample the proportion of mutant DNA is measured.
bronzing > Making 45 samples we recorded a mean proportion of mutant of
29.1% with a standard deviation of 3.5%.
> > >
RF > Since you're treating these samples as samples from the SAME
RF > population, all you have to do is to combine the proportion data
RF > into ONE single estimate of the unknown proportion p,
> > >
RF > p-hat = (total of mutant DNAs in all 45 samples)/(Total sample
size).
> > >
RF > This p-hat is the APPROPRIATE estimate and test statistic of what
RF > the OP was trying to test by the (inappropriate) mean of the
RF > proportions from 45 samples.
> >
RU > Bob is right, to an extent -- testing the overall proportion
RU > is the fine test, if you are willing to make the assumption
RU > of homogeneity, and if the Ns in each sample are similar.
That's Ulrich re-confirming his own BLUNDER!
>
> ----- snip ------------
>
>
> Here's a question for bronzing: Why did you collect 45 samples instead
> of one large sample?
bronzing can answer for himself if he is still reading this ng, but
I suspect it was the element of time/space that he was sampling
from a large area (or a small area at different times) that was
considered by him to be a single population.
bronzing heard of the t-test. Every non-statistician heard of
the t-test, and most of them use it improperly. This was such
an example.
bronzing obvious didn't realize that all he had to do was to use
the information from his 45 samples to get one PROPORTION estimate.
>
> --
> Bruce Weaver
> bwe...@lakeheadu.ca
> www.angelfire.com/wv/bwhomedir
Not content to re-confirm his own ERRORS and Quackery, Richard
Ulrich added,
RU> My advice is based on 30 years of experience, doing research
RU> and giving advice to researchers.
Confession of 30 years of Quackery by the Quack.
RU> Bob's test of p is the ivory-tower solution, with problems
RU> that I named.
Ad hominem attack based on Ulrich's OWN errors!
Ulrich's final dig, after I caught him with more nonsense:
RF> Irrelevant NONSENSE for the OP's problem in question. A
RF> T-test is a completely inappropriate test for p = 0.5.
Ulrich> Probably irrelevant for the OP; far from nonsense.
That's bad enough, spewing irrelevance to the OP,
Ulrich> Bob might look up "one-sample t-test," if that is his problem.
To Jerry Dallal: I think your posting time will be much
better served and will benefit this community considerably
if you would step in and correct the ERRORS, MISINFORMATION,
and QUACKERY committed by the resident Quack Richard Ulrich.
To all other readers and posters of this ng: In cases of
OBVIOUS errors by Richard Ulrich, on matters of testing a
single proportion to advise the OP to use a t-test, for
entirely wrong reasons, that you should step in, as a Good
Citizen of the community to CORRECT him?
Richard Ulrich's blatent errors, and his even worse attempt
to re-assert his own errors as appropriate, likens a
street thug, having been caught red-handed by a cop for
mugging, turned around and tried to mug the cop! <;^)
-- Bob.
than arguing
Radford Neal
> Just to clarify my problem (coz I'm not sure if you got me right and
> the proposed procedures are applicable): We made 45 gels from tissue
> samples of one patient which contain partially mutant partially
> wild-type DNA. From the IODs (integrated optical desities) of the
> bands on the gels we calculated for each of the 45 gels the
> proportion of mutant DNA contained in the gel. The mean value of all
> the 45 proportions is 0.291 with a standard deviation of 0.035. The
> proposed web-site deals with samples taken from populations, each
> sample producing an integer number. (Am I wrong?) I don't know how
> this is comparible to my problem. I posted the same question to the
> forim of OriginLab (the software I'm using for my
> calculations). Here's the answer I got until now. What do you think?
You should just do a t test. Your proportions are not integer counts
out of some integer total, so any method designed for that situation
is inapplicable. In theory, the t test assumes that the values are
normally distributed, but it's robust to departures from this
assumption, especially if the sample size isn't too small (45 is big
enough), and the departures from normalily don't take the form of
occassional extreme values. Since your values are between 0 and 1,
there can't be any really extreme values.
The more crucial assumption is that the values are independent.
There's no obvious reason to think this assumption is violated from
your description, but it's something to keep in mind.
Ignore the advice from "Reef Fish", who is a raving idiot.
----------------------------------------------------------------------------
Radford M. Neal rad...@cs.utoronto.ca
Dept. of Statistics and Dept. of Computer Science rad...@utstat.utoronto.ca
University of Toronto http://www.cs.utoronto.ca/~radford
----------------------------------------------------------------------------
Scott Seidman
news:2005Jul27....@jarvis.cs.toronto.edu:
> In article <42e7512f$0$16686$9b4e...@newsread2.arcor-online.net>,
> <bron...@net.hr> wrote:
>
>> Just to clarify my problem (coz I'm not sure if you got me right and
>> the proposed procedures are applicable): We made 45 gels from tissue
>> samples of one patient which contain partially mutant partially
>> wild-type DNA. From the IODs (integrated optical desities) of the
>> bands on the gels we calculated for each of the 45 gels the
>> proportion of mutant DNA contained in the gel. The mean value of all
>> the 45 proportions is 0.291 with a standard deviation of 0.035. The
>> proposed web-site deals with samples taken from populations, each
>> sample producing an integer number. (Am I wrong?) I don't know how
>> this is comparible to my problem. I posted the same question to the
>> forim of OriginLab (the software I'm using for my
>> calculations). Here's the answer I got until now. What do you think?
>
> You should just do a t test. Your proportions are not integer counts
> out of some integer total, so any method designed for that situation
> is inapplicable. In theory, the t test assumes that the values are
> normally distributed, but it's robust to departures from this
> assumption, especially if the sample size isn't too small (45 is big
> enough), and the departures from normalily don't take the form of
> occassional extreme values. Since your values are between 0 and 1,
> there can't be any really extreme values.
>
A value of 1.0 is 20 SD's off his mean!!
--
Scott
Reverse name to reply
bron...@net.hr
"Reef Fish" <Large_Nass...@Yahoo.com> schrieb:
.. snip ...
>
> If p(i)-hat= k(i)/N(i), i = 1, ..., 45, everyone SHOULD know the
> combined proportion is NOT Sum<i=1,...45> (p(i)-hat)/45, for
> precisely the reason that the N(i)s may be considerably different.
> Ulrich's use of the word "homogeneous" is wrong on two counts --
> (a) wrong word for different sample sizes; (b) for his reason.
>
> That's why the combined proportion estimate is
>
> p-hat = (sum<i> k(i))/(sum<i> N(i)).
Dear Bob,
it's possible we are taking about different things here. I don't have any k(i) and N(i).
One sample consists just of taking one electrophoseris gel and deviding the intesities
of one band through the sum of the intensities of same band and another one.
This yealds a number between 0 and 1 for each sample (representing the proportion
of mutant DNA). The mean of the thus measured values is 0.291 with a SD of 0.035.
Are you still against the t-test?
Bye
Bronzing
Reef Fish
That's beside the point. Radford has proven himself to know even
less about statistics than Richard Ulrich (which is hard to believe).
Radford's statement "Since your values are between 0 and 1,
there can't be any really extreme values" is an indication of
his extreme naivite in statistics!
This thread is fragmented between sci.stat.math and sci.stat.edu,
but should be in BOTH.
-- Bob.
Radford Neal
>> <bron...@net.hr> wrote:
>>
>>> Just to clarify my problem (coz I'm not sure if you got me right and
>>> the proposed procedures are applicable): We made 45 gels from tissue
>>> samples of one patient which contain partially mutant partially
>>> wild-type DNA. From the IODs (integrated optical desities) of the
>>> bands on the gels we calculated for each of the 45 gels the
>>> proportion of mutant DNA contained in the gel. The mean value of all
>rad...@cs.toronto.edu (Radford Neal) wrote in
>news:2005Jul27....@jarvis.cs.toronto.edu:
>> You should just do a t test. ... Since your values are between 0 and 1,
>> there can't be any really extreme values.
In article <Xns96A0690EB7CDFsc...@130.133.1.4>,
Scott Seidman <namdie...@mindspring.com> wrote:
>A value of 1.0 is 20 SD's off his mean!!
A good point. Maybe some checking for outliers would indeed be in order.
Still, one reason to be concerned about extreme points is that they
indicate that even more extreme points might also be possible (even if
they didn't appear in the sample). Having absolute bounds certainly
helps alleviate that concern.
Reef Fish
Because of the fact that the above didn't appear in sci.stat.edu until
this morning, and the thread had been fragmented, the above was unknown
to me.
All I relied on what the fact that you had 45 SAMPLE PROPORTIONS, in
your
OP and that cited by Ulrich!
Given your clarification, two additional ideas seem relevant:
1. If you think what's measure CAN be interpreted as a "proportion",
then
there should be some way for you to judge (or estimate) the
APPROXIMATE "count" and "total count" that constituted the
"proportion", through the properties of your measuring instrument.
2. If you can't come up with the COUNTS or estimated counts, then
you'll have a much less trustworthy estimate of the overall
proportion, but the "mean" is still a "proportion", possibly
not best or not appropriate with your data (though not
necessarily so, if each measurement has the same precision
to be considered having a common "total size" N(i) in each
of the 45 measurements).
In any event, as stated by you, regarding the background of your
data, it is a problem in testing PROPORTIONS, not a problem in
testing the MEAN of 45 pieces of data, each coming from a different
population (the different bands of your measuring instrument).
-- Bob.
m00es
Very very true!
But it's an interesting problem in its own right, so let's try to tease
it apart (CAUTION: rant with uncessary details follows).
The main question is: Is the proportion of mutant DNA equal to .5?
Denote the true proportion of mutant DNA as phi.
Now, one could take a sample of size n and calculate the proportion of
mutant DNA in that sample (call that p). Under H0: phi = .5, we know
that
p ~ N(.5, .5(1-.5)/n)
based on the properties of the binomial distribution. The typical
method to test H0 is to calculate
z = (p - .5)/(sqrt(.5(1-.5)/n),
which, under H0, follows approximately a standard normal distribution.
Now, it seems we have a problem here, because we don't know what n in a
single sample is! It's one of those situations where all you can say
is: "it's 30% mutant and 70% other" and that's it -- just like you can
fill a glass with 30% milk and 70% water and wouldn't know n in that
case either.
Okay, so we can take another approach. Take i = 1, ..., 45 separate
proportions. Denote them as p_1, ..., p_45. Under H0,
p_i ~ N(.5, .5(1-.5)/n_i).
Now, I think it is perfectly reasonable to calculate
(bar(p) - .5) / sd(p)/sqrt(45),
where bar(p) is the mean of the proportions and sd(p) the standard
deviation of the 45 proportions, and just compare that against a
standard normal distribution. For conservativeness, we could use a
t-distribution, but with df = 44, it makes little difference.
Now, I see two issues:
1) The 45 samples are taken from the same individual. So, can we simply
treat p_1, ..., p_45 as independent? Maybe tissue samples taken from
regions closer to each other are more alike than samples from regions
further apart. Then we get dependencies between the proportions. This
is more of a question for a biologist to answer.
2) Can we assume that phi is equal to .5 for all 45 samples? If phi can
be assumed not to vary between the samples (again, a question for a
biologist), then all is fine. However, take the case where phi varies.
Then we really have phi_1, ..., phi_45 and each p_i actually comes from
a different distribution. To go further, we may need some additional
distributional assumptions. Assume that the phi values are actually
sampled from a normal distribution with expected value mu_phi and
variance var_phi. Therefore:
phi_i ~ N(mu_phi, var_phi)
pi ~ N(phi_i, phi_i(1-phi_i)/n_i)
Now what we would test is whether the expected value of the phi values
(mu.phi) is equal to .5 or not. Assume that the n_i's are constant
across the samples (all equal to n). Plus, we apply the arcsine
transformation. Then we have:
p*_i = arcsine(sqrt(p_i)) ~ N(arcsine(sqrt(phi_i)), 1/(4n)).
The optimally weighted average of the p*_i values uses w_i = 1/(var_phi
+ 1/4n) as its weights. We still don't know n (or var_phi), but that's
not problem, because the weights are all equal to each other, so the
weighted average of the p*_i values is equal to the simple average.
Then we are back to
(bar(p*) - .5) / sd(p*)/sqrt(45),
this time using the transformed values, which now tests H0: mu_phi = 0.
Without the arcsine transformation, you get heterogeneous sampling
variances and this approach isn't quite applicable anymore. Of course,
the assumption of equal n's also needs to hold.
This problem actually calls on methods frequently used in
meta-analysis. Whether the phi values are homogeneous or heterogeneous
gets into the same issue as whether to apply a fixed- or a
random-effects model in meta-analysis.
m00es
Reef Fish
m00es wrote:
> > So in this case, it's probably not worth splitting hairs about how many angels > can dance on the head of a pin.
>
> Very very true!
>
> But it's an interesting problem in its own right, so let's try to tease
> it apart (CAUTION: rant with uncessary details follows).
Your "uncessary" (interesting Freudian slip or newly coined word)
details are both accurate and necessary. :-) A good harey
discussion of the technical points that obviously escaped most
of the hare- or hair-splitters, and the block of flind sheep
grazing in this pasture. :) Howzat for a mixed metaphor,
simile, and quasi-spoonerisms.
-- Bob.
m00es
> Then we are back to
>
> (bar(p*) - .5) / sd(p*)/sqrt(45),
>
> this time using the transformed values, which now tests H0: mu_phi = 0.
This, of course, should have read H0: mu_phi = .5.
m00es
DZ
> "m00es" <m0...@yahoo.com> schrieb:
>> A variance stabilizing transformation (in this particular case, the
>> arcsin transformation) is often applied before analyzing
>> proportions in such a manner. Therefore, take each proportion and
>> apply arcsin( sqrt(p) ) which removes, approximately, the
>> dependency of the variance on the mean.
>
Without going into appropriateness of that and while abusing some
notation, we have X ~ Bin(n, p) with E(X/n) = p; V(X/n) = 1/n * p(1-p)
Denote f(x) = arcsin(sqrt(x))
then by the Taylor series approximation V(f(X/n)) is approximately
(f'(p))^2 V(X/n) = 1/n * 1/4
so it doesn't depend on p.
DZ
Richard Ulrich
concerning ideas and issues that arose in several other posts.
One detail was added about the data, that these were 45
'assays' on one individual, where there are no simple counts
hiding behind the proportions.
In my initial Reply, I focused mainly on the idea that the
t-test is rather robust, and there was no evident reason why
it could not be used -- I will add other comments at the
bottom of that reply, duplicated here for context.
===== end of post and my first reply.
T-tests can be used rather robustly, as I stated. Frederick Lord
offered his famous example in the 1950s, using t-test to
analyze distributions of numbers on team jerseys. Analyzing
numerical proportions between 20% and 80% isn't highly
controversial.
1. PROPORTIONS.
Proportions sometimes offer other opportunities and hazards.
For instance, based on counts, there could be 45 samples that
average 33% mutant. For *certain* hypotheses and assumptions,
it could be fair and proper to aggregate counts over all 45 samples.
This aggregation problem put me in mind of meta-analyses.
But "meta-analyses" face the aggregation problem by insisting
at the start that there be homogeneity of means, or else there
is great concern about what the average might mean. For
instance, if heterogeneous, *what* is the source of heterogeneity?
There could be an overall proportion of 33% if 15 of the samples
were 100% wild and 30 of the samples were 100% mutant.
That would have a far wider Confidence Interval than the
opposite extreme, where each of the 45 samples reports
almost precisely 33% wild.
The "aggregation" answer assumes that there is a sort of
randomness acting across all the samples, and that (in
meta-analysis) one has made a choice that each sample is
weighted equally.
2. TRANSFORMATION OF PROPORTIONS (like arc-sine).
The purpose of transforming proportions is to achieve
equal weighting in ANOVA. The arc-sine of sqrt(p) does
that for counts that are binomially distributed -- it does not
necessarily do it for "proportions" measured in other fashions.
That transformation was popular in the days before computers,
I think, because it allowed quick and reliable computation
by research assistants. (In simulations that I have run, it gave
p-values that were too small, for testing between low proportions.)
I can't claim to know anything about the wild and mutant
genes, but I found this possibly-relevent comment on the web at
http://www-users.med.cornell.edu/~jawagne/screening_for_mutations.html
"Heteroduplex Analysis (HET).
[ snip, detail]
"Limit of detection: Ratios of mutant to wild-type
DNA of < 1:5 may not be detectable. "
This tells me that the OPs observed CI of (22,36) for the
set of 45 observations *might* be approaching the lower
limit of what is measurable. I don't know how that might
affect the present problem, except that it should rule out
the rationality of the arc-sine sqrt(p) for any similar data.
3. ANOTHER COMMENT ON THE ORIGINAL POST.
The observed distribution of means includes variation
for "measurement" and variation for "sample" and it
still achieved a rather narrow range of results -- mean (SD)
equal to 29.1 (3.5). What is the variation for a split-sample
test, if that can be done?
All 45 were drawn from one individual. Is this high
consistency, SD= 3.5, something that is unexpected?
Is the wide divergence from 50% something that is
unexpected? I'm suggesting, here, that getting a t-test
of t= 40.0 is such an extreme result that it raises the question
of why the sample size was so large. Well, maybe it
is just as cheap and easy to do 45 as doing 5, but I'm
one reader who doesn't know that.
If a range that narrow is predictable, then it was not
really necessary to look at 45 measurements, for a
difference of 29% vs. 50%.
4. REASONS NOT TO USE T-TESTS FOR SIMILAR PROBLEMS.
What does the "45" represent?
Is that 45 combinations of location (foot, torso, head, ...)
and cell type (skin, bone, muscle, smooth muscle, ...)?
A couple of posters commented that they should want
to know more about the problem before saying that a
t-test was always right. Well, here are two of the questions
that might be asked, or one question with two parts.
Are the 45 samples independent numbers that should
represent the same number, or does something like "cell type"
or "location" represent a source of consistency? If "bones"
may present a different average than "skin", there are two
consequences.
- First, the general hypothesis needs more careful specification.
WHAT is supposed to be less than 50%?
I would guess that the overall mean should be computed
to represent an intentional weighting -- perhaps that was
done at the start, in selecting the 45 samples.
- Second, the overall error term is different from that of
the t-test. It could be smaller, if the form of the general
hypothesis doesn't get more complicated.
Hope this was interesting.