Due to sample size calculation for cut-off values
695 views
Skip to first unread message
beisikas
Jul 13, 2011, 2:25:23 PM7/13/11
to MedStats
Hi all.
I am a newbe in this group, but not a newbe in medical statistics :)
Unfortunately, my knowledge is to weak in order to find the right way
calculating sample size.
My work is as follows. In healthy me, I will examine the concentration
of some Marker serum (lets call this variable "MarkerX", and also some
other marker (that is predictor of somatic disease, lets call it
"MarkerY"). For known reasons, I can't tell all the details :) But
both variables, MarkerX and MarkerY are continues. So, all the men
will be classified according the MarkerY values as "Positive" (the
value of MarkerY is above normal range, big risk for disease) and
"Negative" (the value of MarkerY is within the normal range, t.i.,
small risk for disease). Comparisons between these two groups will be
made (MarkerX concentration means, ajusting for confounders and so on)
and logistic regresion models will be created.
The mean aim of this study is try to find the "cut-off" value of
MarkerX. T.i., to found out where is the thereshold, when the risk for
having disease (which is showed by increasing MarkerY value)
significantly rises if the patient has decreased MarkerX value.
Analysis is going to be made usign ROC curves, I am familiar with
this :)
But how to calculate the sample size for this study? I found that
special complex formulas should be used, dealing with ROC curves or
smth., but I don't know exactly what kind of.
Some info is at http://www.compass.fhcrc.org/edrnnci/files/pdf/PepeSSCalc.pdf,
but I guess it isn't what I need.
Thanks for any help.
I am a newbe in this group, but not a newbe in medical statistics :)
Unfortunately, my knowledge is to weak in order to find the right way
calculating sample size.
My work is as follows. In healthy me, I will examine the concentration
of some Marker serum (lets call this variable "MarkerX", and also some
other marker (that is predictor of somatic disease, lets call it
"MarkerY"). For known reasons, I can't tell all the details :) But
both variables, MarkerX and MarkerY are continues. So, all the men
will be classified according the MarkerY values as "Positive" (the
value of MarkerY is above normal range, big risk for disease) and
"Negative" (the value of MarkerY is within the normal range, t.i.,
small risk for disease). Comparisons between these two groups will be
made (MarkerX concentration means, ajusting for confounders and so on)
and logistic regresion models will be created.
The mean aim of this study is try to find the "cut-off" value of
MarkerX. T.i., to found out where is the thereshold, when the risk for
having disease (which is showed by increasing MarkerY value)
significantly rises if the patient has decreased MarkerX value.
Analysis is going to be made usign ROC curves, I am familiar with
this :)
But how to calculate the sample size for this study? I found that
special complex formulas should be used, dealing with ROC curves or
smth., but I don't know exactly what kind of.
Some info is at http://www.compass.fhcrc.org/edrnnci/files/pdf/PepeSSCalc.pdf,
but I guess it isn't what I need.
Thanks for any help.
Steve Simon, P.Mean Consulting
Jul 13, 2011, 5:01:32 PM7/13/11
to meds...@googlegroups.com, beisikas
On 7/13/2011 1:25 PM, beisikas wrote:
> My work is as follows. In healthy me, I will examine the concentration
> of some Marker serum (lets call this variable "MarkerX", and also some
> other marker (that is predictor of somatic disease, lets call it
> "MarkerY"). For known reasons, I can't tell all the details :) But
> both variables, MarkerX and MarkerY are continues. So, all the men
> will be classified according the MarkerY values as "Positive" (the
> value of MarkerY is above normal range, big risk for disease) and
> "Negative" (the value of MarkerY is within the normal range, t.i.,
> small risk for disease). Comparisons between these two groups will be
> made (MarkerX concentration means, ajusting for confounders and so on)
> and logistic regresion models will be created.
>
> The mean aim of this study is try to find the "cut-off" value of
> MarkerX. T.i., to found out where is the thereshold, when the risk for
> having disease (which is showed by increasing MarkerY value)
> significantly rises if the patient has decreased MarkerX value.
> Analysis is going to be made usign ROC curves, I am familiar with
> this :)
>
> But how to calculate the sample size for this study? I found that
> special complex formulas should be used, dealing with ROC curves or
> smth., but I don't know exactly what kind of.
> Some info is at
http://www.compass.fhcrc.org/edrnnci/files/pdf/PepeSSCalc.pdf,
> but I guess it isn't what I need.
You will get a lot of comments along the lines of "you shouldn't analyze
it this way because..." and I'd encourage you to look at those comments
carefully and with an open mind.
But to directly answer your question without the distraction of changing
your research direction 180 degrees, I would suggest this:
If the sample size formulas associated with ROC curves are too messy or
too unclear to implement, then why not establish that sensitivity and
specificity both have reasonably narrow confidence intervals? You won't
know the sensitivity and specificity until you do the research, of
course, but set up a plausible scenario (or if you're really ambitious a
range of plausible scenarios) and then calculate the widths of the
confidence intervals. If your intervals are too wide, increase your
sample size until you are happy. In the unlikely event that your
intervals are too narrow, cut back on your sample size.
By the way, you need data on men with disease in order to compute
sensitivity OR to establish a reasonable cutoff OR to compute an ROC
curve. Do you really plan to study ONLY healthy men (I assume that
"healthy me" is a typo and not an indication of self-experimentation).
If so, then you need to radically revise your statistical approach.
Steve Simon, n...@pmean.com, Standard Disclaimer.
Sign up for the Monthly Mean, the newsletter that
dares to call itself average at www.pmean.com/news
beisikas
Jul 14, 2011, 3:50:50 AM7/14/11
to MedStats
Thanks to Mr. Steve.
> By the way, you need data on men with disease in order to compute
> sensitivity OR to establish a reasonable cutoff OR to compute an ROC
> curve. Do you really plan to study ONLY healthy men (I assume that
> "healthy me" is a typo and not an indication of self-experimentation).
> If so, then you need to radically revise your statistical approach.
Some explanation needs to be made here :)
The "healthy" in my study means "generally healthy", t.i., young men
with no complains, no history of present chronic illness, not taking
remedies or food supllements, and aren't smokers. Such a term
"generally healthy" is not so rarely used in medical research and
articles.
And the "disease" perhaps is not the best word to use. These "healthy
men" can have a disturbance, let's call it "dysfunction", or
"functional disturbance" that produces no signs or symptoms (yet), but
could be detected by using biomarkers, such as MarkerY in my study,
from blood samples. The dysfunction, if not corrected, can lead to a
clinically clear disease. So, in general, there is the purpose for a
clinician to reveal, if possible, the disturbance in an early stage,
t.i., in the period of dysfunction, where the patologic changes could
be more easily reversed, or cured. And of course, performing research
on "generally healthy" people (not on "diseased") helps to minimize
the possiblity of influence of additional confounders, such as acute
or chronic inflamational disease or remedies being taken.
There were some studies performed, where the young "healthy" men (of
similar ages as in my study) where examed and from the published
articles I could get some data the prevalence (or other statistical
data) of that "dysfunction" discussed above, in case I need this data
for some formula.
Regards,
A.Bleizgys
> By the way, you need data on men with disease in order to compute
> sensitivity OR to establish a reasonable cutoff OR to compute an ROC
> curve. Do you really plan to study ONLY healthy men (I assume that
> "healthy me" is a typo and not an indication of self-experimentation).
> If so, then you need to radically revise your statistical approach.
The "healthy" in my study means "generally healthy", t.i., young men
with no complains, no history of present chronic illness, not taking
remedies or food supllements, and aren't smokers. Such a term
"generally healthy" is not so rarely used in medical research and
articles.
And the "disease" perhaps is not the best word to use. These "healthy
men" can have a disturbance, let's call it "dysfunction", or
"functional disturbance" that produces no signs or symptoms (yet), but
could be detected by using biomarkers, such as MarkerY in my study,
from blood samples. The dysfunction, if not corrected, can lead to a
clinically clear disease. So, in general, there is the purpose for a
clinician to reveal, if possible, the disturbance in an early stage,
t.i., in the period of dysfunction, where the patologic changes could
be more easily reversed, or cured. And of course, performing research
on "generally healthy" people (not on "diseased") helps to minimize
the possiblity of influence of additional confounders, such as acute
or chronic inflamational disease or remedies being taken.
There were some studies performed, where the young "healthy" men (of
similar ages as in my study) where examed and from the published
articles I could get some data the prevalence (or other statistical
data) of that "dysfunction" discussed above, in case I need this data
for some formula.
Regards,
A.Bleizgys
beisikas
Jul 14, 2011, 3:58:38 AM7/14/11
to MedStats
By the way, I am of course familiar with sensitivity and specificity,
and CIs. But maybe You could advice some valued resourses dealing with
the things I need, t.i. sample size calculation by using CI and so on?
You may agree that for a clinician (at present I work as a physician,
family doctor) biomedical statistics is not an easy thing :)
and CIs. But maybe You could advice some valued resourses dealing with
the things I need, t.i. sample size calculation by using CI and so on?
You may agree that for a clinician (at present I work as a physician,
family doctor) biomedical statistics is not an easy thing :)
Andrius Bleizgys
Jul 14, 2011, 4:25:39 AM7/14/11
to meds...@googlegroups.com
And one more explanation could be necessary.
At present, the cut-off value for MarkerX is, say, 20 units/ml, that means,
that all people that have MarkerX concentration in blood below 20 U/ml, are
"insufficient" and at high risk for developing some diseases. At those
having MarkerX >= 20 U/ml are sufficient and at low risk.
Some articles suggest that the thereshold of 20 U/ml is to small, and the
normal value of MarkerX is to be at least 40 U/ml, and I am tending to agree
with those researchers. Of course, they used not MarkerX in analyses and
making such a conclusion, they used some other clinical and epidemiological
data.
So, I have the purpose to prove with my research that the cut-off value for
MarkerX is 40 U/ml, t.i., 40-20 = 20 U/ml (the proposed difference), maybe I
could try some formulas (for sample size calculation) according to "means"?
The means or SD of MarkerX in the populiation of healthy yuong men in my
country is also available in the literature.
And if I think in right way, I calculate sample size using mean's formula,
and having performed research, get the results that cut-off value is not 40
(as I suggested). but say 20 or 10, or 55 U/ml? Will my research in such a
case be considered as having to less power or statistically "not well
designed", and so on?
At present, the cut-off value for MarkerX is, say, 20 units/ml, that means,
that all people that have MarkerX concentration in blood below 20 U/ml, are
"insufficient" and at high risk for developing some diseases. At those
having MarkerX >= 20 U/ml are sufficient and at low risk.
Some articles suggest that the thereshold of 20 U/ml is to small, and the
normal value of MarkerX is to be at least 40 U/ml, and I am tending to agree
with those researchers. Of course, they used not MarkerX in analyses and
making such a conclusion, they used some other clinical and epidemiological
data.
So, I have the purpose to prove with my research that the cut-off value for
MarkerX is 40 U/ml, t.i., 40-20 = 20 U/ml (the proposed difference), maybe I
could try some formulas (for sample size calculation) according to "means"?
The means or SD of MarkerX in the populiation of healthy yuong men in my
country is also available in the literature.
And if I think in right way, I calculate sample size using mean's formula,
and having performed research, get the results that cut-off value is not 40
(as I suggested). but say 20 or 10, or 55 U/ml? Will my research in such a
case be considered as having to less power or statistically "not well
designed", and so on?
Thanks in advance.
Neil Shephard
Jul 14, 2011, 4:35:17 AM7/14/11
to meds...@googlegroups.com
On Thu, Jul 14, 2011 at 8:58 AM, beisikas <andrius....@gmail.com> wrote:
> But maybe You could advice some valued resourses dealing with
> the things I need, t.i. sample size calculation by using CI and so on?
> But maybe You could advice some valued resourses dealing with
> the things I need, t.i. sample size calculation by using CI and so on?
This reference may be useful...
@article{bland2009,
author = {Bland, J. M.},
citeulike-article-id = {6128105},
citeulike-linkout-0 = {http://dx.doi.org/10.1136/bmj.b3985},
day = {6},
doi = {10.1136/bmj.b3985},
issn = {1468-5833},
journal = {BMJ},
keywords = {methodology, power, samplesize, statisics},
month = oct,
number = {oct06 3},
pages = {b3985},
posted-at = {2010-03-30 13:23:21},
priority = {0},
title = {{The tyranny of power: is there a better way to calculate
sample size?}},
url = {http://dx.doi.org/10.1136/bmj.b3985},
volume = {339},
year = {2009}
}
A version is available from Martin Blands home page at
http://www-users.york.ac.uk/~mb55/talks/tyrpowertalk.pdf if you do not
have access to the BMJ
(http://www.bmj.com/content/339/bmj.b3985.extract)
Neil
--
“Truth in science can be defined as the working hypothesis best suited
to open the way to the next better one.” - Konrad Lorenz
Email - nshe...@gmail.com
Website - http://kimura.no-ip.org/
Photos - http://www.flickr.com/photos/slackline/
MartinHolt
Jul 14, 2011, 10:22:37 AM7/14/11
to MedStats
Hi Andrius,
If you are to go down the confidence interval, or Bayesian credible
approach, there will be no adjustment for potential confounders. So
carrying on in that vein, with both markers X and Y being continuous,
it might be interesting to plan to plot X values versus Y values in
the form of a "functional relationship". (Also known as "Deming
regression", both x and y axes are allowed to vary, as here, whereas
in a normal plot only the x-axis can vary).
On completion of the study there will be four possible sets of data:
1. X value < cut-off, Y value > cut-off
2. X value < cut-off, Y value <= cut-off
3. X value >= cut-off, Y value > cut-off
4. X value <= cut-off, Y value <= cut-off
These data could be presented in the usual 2x2 format for calculating
sensitivity, specificity and prevalence.
The link below shows how one can turn this around to calculate sample
size based on choosing values for the cut-offs and for sensitivity,
specificity, and prevalence. You input expected or desired values for
sensitivity, specificity, prevalence, desired confidence interval
(e.g. 95% or 90%, etc) AND the desired precision of the test. The link
calculates sample sizes based on (a) the desired precision for
sensitivity, (b) that of specificity and finally (c) both combined.
I believe that the derivation of the formulae is based on using the
width of the appropriate confidence interval to generate the desired
precision figure.
http://www.google.co.uk/#hl=en&sugexp=esqb,ratio&xhr=t&q=sensitivity+and+specificity+sample+size+calculation&cp=40&pf=p&sclient=psy&source=hp&aq=0&aqi=g1g-b1&aql=&oq=sensitivity+and+specificity+sample+size+&pbx=1&bav=on.2,or.r_gc.r_pw.&fp=a53fd02e665c50e5&biw=1040&bih=874
I think you can play with the numbers to generate meaningful targets
for sensitivity, specificity and prevalence without worrying about
multiple testing (e.g. Bonferroni) as (following Professor Bland's
paper recommended above) this method does not use "power" or
"significance" but is based on the confidence interval for a
proportion.
Hope that helps,
Martin Holt
Medical Statistician
On Jul 14, 9:35 am, Neil Shephard <nsheph...@gmail.com> wrote:
> Website -http://kimura.no-ip.org/
> Photos -http://www.flickr.com/photos/slackline/
If you are to go down the confidence interval, or Bayesian credible
approach, there will be no adjustment for potential confounders. So
carrying on in that vein, with both markers X and Y being continuous,
it might be interesting to plan to plot X values versus Y values in
the form of a "functional relationship". (Also known as "Deming
regression", both x and y axes are allowed to vary, as here, whereas
in a normal plot only the x-axis can vary).
On completion of the study there will be four possible sets of data:
1. X value < cut-off, Y value > cut-off
2. X value < cut-off, Y value <= cut-off
3. X value >= cut-off, Y value > cut-off
4. X value <= cut-off, Y value <= cut-off
These data could be presented in the usual 2x2 format for calculating
sensitivity, specificity and prevalence.
The link below shows how one can turn this around to calculate sample
size based on choosing values for the cut-offs and for sensitivity,
specificity, and prevalence. You input expected or desired values for
sensitivity, specificity, prevalence, desired confidence interval
(e.g. 95% or 90%, etc) AND the desired precision of the test. The link
calculates sample sizes based on (a) the desired precision for
sensitivity, (b) that of specificity and finally (c) both combined.
I believe that the derivation of the formulae is based on using the
width of the appropriate confidence interval to generate the desired
precision figure.
http://www.google.co.uk/#hl=en&sugexp=esqb,ratio&xhr=t&q=sensitivity+and+specificity+sample+size+calculation&cp=40&pf=p&sclient=psy&source=hp&aq=0&aqi=g1g-b1&aql=&oq=sensitivity+and+specificity+sample+size+&pbx=1&bav=on.2,or.r_gc.r_pw.&fp=a53fd02e665c50e5&biw=1040&bih=874
I think you can play with the numbers to generate meaningful targets
for sensitivity, specificity and prevalence without worrying about
multiple testing (e.g. Bonferroni) as (following Professor Bland's
paper recommended above) this method does not use "power" or
"significance" but is based on the confidence interval for a
proportion.
Hope that helps,
Martin Holt
Medical Statistician
On Jul 14, 9:35 am, Neil Shephard <nsheph...@gmail.com> wrote:
> On Thu, Jul 14, 2011 at 8:58 AM, beisikas <andrius.bleiz...@gmail.com> wrote:
> > But maybe You could advice some valued resourses dealing with
> > the things I need, t.i. sample size calculation by using CI and so on?
>
> This reference may be useful...
>
> @article{bland2009,
> author = {Bland, J. M.},
> citeulike-article-id = {6128105},
> citeulike-linkout-0 = {http://dx.doi.org/10.1136/bmj.b3985},
> day = {6},
> doi = {10.1136/bmj.b3985},
> issn = {1468-5833},
> journal = {BMJ},
> keywords = {methodology, power, samplesize, statisics},
> month = oct,
> number = {oct06 3},
> pages = {b3985},
> posted-at = {2010-03-30 13:23:21},
> priority = {0},
> title = {{The tyranny of power: is there a better way to calculate
> sample size?}},
> url = {http://dx.doi.org/10.1136/bmj.b3985},
> volume = {339},
> year = {2009}
>
> }
>
> A version is available from Martin Blands home page athttp://www-users.york.ac.uk/~mb55/talks/tyrpowertalk.pdfif you do not
> > But maybe You could advice some valued resourses dealing with
> > the things I need, t.i. sample size calculation by using CI and so on?
>
> This reference may be useful...
>
> @article{bland2009,
> author = {Bland, J. M.},
> citeulike-article-id = {6128105},
> citeulike-linkout-0 = {http://dx.doi.org/10.1136/bmj.b3985},
> day = {6},
> doi = {10.1136/bmj.b3985},
> issn = {1468-5833},
> journal = {BMJ},
> keywords = {methodology, power, samplesize, statisics},
> month = oct,
> number = {oct06 3},
> pages = {b3985},
> posted-at = {2010-03-30 13:23:21},
> priority = {0},
> title = {{The tyranny of power: is there a better way to calculate
> sample size?}},
> url = {http://dx.doi.org/10.1136/bmj.b3985},
> volume = {339},
> year = {2009}
>
> }
>
> have access to the BMJ
> (http://www.bmj.com/content/339/bmj.b3985.extract)
>
> Neil
>
> --
> “Truth in science can be defined as the working hypothesis best suited
> to open the way to the next better one.” - Konrad Lorenz
>
> Email - nsheph...@gmail.com
> (http://www.bmj.com/content/339/bmj.b3985.extract)
>
> Neil
>
> --
> “Truth in science can be defined as the working hypothesis best suited
> to open the way to the next better one.” - Konrad Lorenz
>
> Website -http://kimura.no-ip.org/
> Photos -http://www.flickr.com/photos/slackline/
beisikas
Jul 14, 2011, 1:44:09 PM7/14/11
to MedStats
Thanks to all who replied.
I would like also to give some additional info.
The type of my study is neither case-control nor cohort. It would be
just cross-sectional, the same men will be examed two times, that is
because the X marker differs significantly during the year. So the
examination will be carried out when the the value of X marker is
expected to be the lowest (first examination) and when the highest
(second examination), in other words, at different season, and we
expect find an inverse assotiation (and also significant correlation)
with marker Y twice, t.i. from the data of first examination and from
data of second one. So in fact there will be two paired samples, but I
don't want to make conclusions about cut-off values for X marker by
comparing these two groups together or smth. We tend to analyse these
"groups" separately, so that's why the such formulas like calculating
sample size for "comparing means between groups" is not the right
thing.
The Y marker is also continuous variable, but I am not interesting in
the cut-off values for it. All the men after study will be classified
into to groups - "0" as having "normal Y marker values" and "1" as
having "increased Y marker values", according to the recomendations
from the reagents provider; there are written instruction, what values
are called "normal" and what - "increased". Of course, it is difficult
to prognose how many men will be normal and how many "not normal" :) I
will used logistic regression model, when the dependent variable is
logit (probability of being in "1" group)/(1-probability being in "0"
group), and independent - as dichotomic "dummy" transformed from
marker X (trying different cut-off values for marker X, so all the men
will be separated into one group with marker X "below cut-off value"
and another group when marker X is "above cut-off value"), and then
prediction according each logistic model, ROC curves analysis and so
on.
On 14 Lie, 17:22, MartinHolt <martinhol...@yahoo.com> wrote:
> Hi Andrius,
>
> If you are to go down the confidence interval, or Bayesian credible
> approach, there will be no adjustment for potential confounders. So
> carrying on in that vein, with both markers X and Y being continuous,
> it might be interesting to plan to plot X values versus Y values in
> the form of a "functio- nal relationship". (Also known as "Deming
> > A version is available from Martin Blands home page athttp://www-users.york.ac.uk/~mb55/talks/tyrpowertalk.pdfifyou do not
I would like also to give some additional info.
The type of my study is neither case-control nor cohort. It would be
just cross-sectional, the same men will be examed two times, that is
because the X marker differs significantly during the year. So the
examination will be carried out when the the value of X marker is
expected to be the lowest (first examination) and when the highest
(second examination), in other words, at different season, and we
expect find an inverse assotiation (and also significant correlation)
with marker Y twice, t.i. from the data of first examination and from
data of second one. So in fact there will be two paired samples, but I
don't want to make conclusions about cut-off values for X marker by
comparing these two groups together or smth. We tend to analyse these
"groups" separately, so that's why the such formulas like calculating
sample size for "comparing means between groups" is not the right
thing.
The Y marker is also continuous variable, but I am not interesting in
the cut-off values for it. All the men after study will be classified
into to groups - "0" as having "normal Y marker values" and "1" as
having "increased Y marker values", according to the recomendations
from the reagents provider; there are written instruction, what values
are called "normal" and what - "increased". Of course, it is difficult
to prognose how many men will be normal and how many "not normal" :) I
will used logistic regression model, when the dependent variable is
logit (probability of being in "1" group)/(1-probability being in "0"
group), and independent - as dichotomic "dummy" transformed from
marker X (trying different cut-off values for marker X, so all the men
will be separated into one group with marker X "below cut-off value"
and another group when marker X is "above cut-off value"), and then
prediction according each logistic model, ROC curves analysis and so
on.
On 14 Lie, 17:22, MartinHolt <martinhol...@yahoo.com> wrote:
> Hi Andrius,
>
> If you are to go down the confidence interval, or Bayesian credible
> approach, there will be no adjustment for potential confounders. So
> carrying on in that vein, with both markers X and Y being continuous,
> it might be interesting to plan to plot X values versus Y values in
> regression", both x and y axes are allowed to vary, as here, whereas
> in a normal plot only the x-axis can vary).
>
> On completion of the study there will be four possible sets of data:
> 1. X value < cut-off, Y value > cut-off
> 2. X value < cut-off, Y value <= cut-off
> 3. X value >= cut-off, Y value > cut-off
> 4. X value <= cut-off, Y value <= cut-off
>
> These data could be presented in the usual 2x2 format for calculating
> sensitivity, specificity and prevalence.
>
> The link below shows how one can turn this around to calculate sample
> size based on choosing values for the cut-offs and for sensitivity,
> specificity, and prevalence. You input expected or desired values for
> sensitivity, specificity, prevalence, desired confidence interval
> (e.g. 95% or 90%, etc) AND the desired precision of the test. The link
> calculates sample sizes based on (a) the desired precision for
> sensitivity, (b) that of specificity and finally (c) both combined.
>
> I believe that the derivation of the formulae is based on using the
> width of the appropriate confidence interval to generate the desired
> precision figure.
>
> http://www.google.co.uk/#hl=en&sugexp=esqb,ratio&xhr=t&q=sensitivity+...
> in a normal plot only the x-axis can vary).
>
> On completion of the study there will be four possible sets of data:
> 1. X value < cut-off, Y value > cut-off
> 2. X value < cut-off, Y value <= cut-off
> 3. X value >= cut-off, Y value > cut-off
> 4. X value <= cut-off, Y value <= cut-off
>
> These data could be presented in the usual 2x2 format for calculating
> sensitivity, specificity and prevalence.
>
> The link below shows how one can turn this around to calculate sample
> size based on choosing values for the cut-offs and for sensitivity,
> specificity, and prevalence. You input expected or desired values for
> sensitivity, specificity, prevalence, desired confidence interval
> (e.g. 95% or 90%, etc) AND the desired precision of the test. The link
> calculates sample sizes based on (a) the desired precision for
> sensitivity, (b) that of specificity and finally (c) both combined.
>
> I believe that the derivation of the formulae is based on using the
> width of the appropriate confidence interval to generate the desired
> precision figure.
>
MartinHolt
Jul 14, 2011, 2:05:05 PM7/14/11
to MedStats
Hi,
It seems that you have decided on analysis by logistic regression,
which does have the advantage of allowing adjustment for confounders.
So I'll not comment further on design. A straight answer to the sample
size question can be found in MedStats archives (I raised it back
then).
http://groups.google.com/group/medstats/browse_thread/thread/b480435bb9f287d6/fc3575eb813512fa?lnk=gst&q=logistic+regression+sample+size#fc3575eb813512fa
This covers the required sample size for the rarer of the two outcomes
(0 or 1), when exact logistic regression should be used and derivation
based on simulations (Peduzzi paper and Frank Harrell's work).
HTH,
Martin Holt
> > > A version is available from Martin Blands home page athttp://www-users.york.ac.uk/~mb55/talks/tyrpowertalk.pdfifyoudo not
>
> - Show quoted text -
It seems that you have decided on analysis by logistic regression,
which does have the advantage of allowing adjustment for confounders.
So I'll not comment further on design. A straight answer to the sample
size question can be found in MedStats archives (I raised it back
then).
http://groups.google.com/group/medstats/browse_thread/thread/b480435bb9f287d6/fc3575eb813512fa?lnk=gst&q=logistic+regression+sample+size#fc3575eb813512fa
This covers the required sample size for the rarer of the two outcomes
(0 or 1), when exact logistic regression should be used and derivation
based on simulations (Peduzzi paper and Frank Harrell's work).
HTH,
Martin Holt
> > > have access to the BMJ
> > > (http://www.bmj.com/content/339/bmj.b3985.extract)
>
> > > Neil
>
> > > --
> > > “Truth in science can be defined as the working hypothesis best suited
> > > to open the way to the next better one.” - Konrad Lorenz
>
> > > Email - nsheph...@gmail.com
> > > Website -http://kimura.no-ip.org/
> > > Photos -http://www.flickr.com/photos/slackline/- Hide quoted text -
> > > (http://www.bmj.com/content/339/bmj.b3985.extract)
>
> > > Neil
>
> > > --
> > > “Truth in science can be defined as the working hypothesis best suited
> > > to open the way to the next better one.” - Konrad Lorenz
>
> > > Email - nsheph...@gmail.com
> > > Website -http://kimura.no-ip.org/
>
> - Show quoted text -
Frank Harrell
Jul 14, 2011, 4:08:11 PM7/14/11
to MedStats
In addition, I can see no motivation for seeking cutoffs or using
sensitivity or specificity in this context. In addition, it is highly
unlikely that a cutoff even exists. The relationship is much more
likely to be smooth than to be discontinuous.
Frank
On Jul 14, 1:05 pm, MartinHolt <martinhol...@yahoo.com> wrote:
> Hi,
>
> It seems that you have decided on analysis by logistic regression,
> which does have the advantage of allowing adjustment for confounders.
> So I'll not comment further on design. A straight answer to the sample
> size question can be found in MedStats archives (I raised it back
> then).
>
> http://groups.google.com/group/medstats/browse_thread/thread/b480435b...
> > > > Photos -http://www.flickr.com/photos/slackline/-Hide quoted text -
sensitivity or specificity in this context. In addition, it is highly
unlikely that a cutoff even exists. The relationship is much more
likely to be smooth than to be discontinuous.
Frank
On Jul 14, 1:05 pm, MartinHolt <martinhol...@yahoo.com> wrote:
> Hi,
>
> It seems that you have decided on analysis by logistic regression,
> which does have the advantage of allowing adjustment for confounders.
> So I'll not comment further on design. A straight answer to the sample
> size question can be found in MedStats archives (I raised it back
> then).
>
beisikas
Jul 15, 2011, 4:31:10 AM7/15/11
to MedStats
Hi, thanks again to all
I'd like to briefly explain my study again. Maybe it would better help
to understand it.
Subject - generally healthy men, aged 20-40.
The blood samples would be taken two times at different seasons
(autumn and spring), and both marker X and Y values (concentration in
the blood serum) will be evaluated each time. Both are countinuous
variables.
The main aims of the study are:
1. To find out how strong is correlation between Marker X and Marker
Y, the analysis (correlation coefficiensts calculating, linear
regression) will be done in each group separately. The correlation is
expected to be inverse.
2. To prove or do deny that the means of Marker X are statistically
different between groups, t.i. in autumn the mean value is greater
than in spring.
3. The same thing for Marker Y, except that, in contrary, it is
expected to be smaller in autumn.
The 3. and 4. will be done by using such tests like ANOVA and ANCOVA
(there are confounders, like BMI); or nonparametric tests, depending
on distribution type).
According to literature, I highly likely that I will prove the
difference of the means for Marker X, but there is almost no data on
seasonic "fluctuations" for marker Y, so that thing is one of my
biggest interests.
4. If the correlation is tend to be weak, or linear regression model
would be not acceptable, then try to use logistic regression model,
and with the help of them try to maked anlysis in order to find the
cut-off value for X, if possible, as described above in previuos
posts.
One of my friends, working more in the biostatics field, advice me to
use the sample size formula, which is design for correlation
coefficients:
N= ( z[1-α] + z[1-β])^2 / (1/2 ln{(1+q)/(1-q)}) ^2 ) + 3
The [1-α] and [1-β] means subscript, t.i. qauantiles. ^2 means square
power.
The q means expected correlation coefficent.
This formula could be find in "Machin D, Campbell MJ, Fayers PM, Pinol
APY. Sample size tables for clinical studies. Oxford: Blackwell
Science; 1997."
My friend suggest to select smaller q for calculations, and if the
study shows that "real" q is greater then predicted, it does nothing
bad, I mean, the power will not be lost.
Maybe it is the right solution for me?
Thanks in advance.
A.B.
I'd like to briefly explain my study again. Maybe it would better help
to understand it.
Subject - generally healthy men, aged 20-40.
The blood samples would be taken two times at different seasons
(autumn and spring), and both marker X and Y values (concentration in
the blood serum) will be evaluated each time. Both are countinuous
variables.
The main aims of the study are:
1. To find out how strong is correlation between Marker X and Marker
Y, the analysis (correlation coefficiensts calculating, linear
regression) will be done in each group separately. The correlation is
expected to be inverse.
2. To prove or do deny that the means of Marker X are statistically
different between groups, t.i. in autumn the mean value is greater
than in spring.
3. The same thing for Marker Y, except that, in contrary, it is
expected to be smaller in autumn.
The 3. and 4. will be done by using such tests like ANOVA and ANCOVA
(there are confounders, like BMI); or nonparametric tests, depending
on distribution type).
According to literature, I highly likely that I will prove the
difference of the means for Marker X, but there is almost no data on
seasonic "fluctuations" for marker Y, so that thing is one of my
biggest interests.
4. If the correlation is tend to be weak, or linear regression model
would be not acceptable, then try to use logistic regression model,
and with the help of them try to maked anlysis in order to find the
cut-off value for X, if possible, as described above in previuos
posts.
One of my friends, working more in the biostatics field, advice me to
use the sample size formula, which is design for correlation
coefficients:
N= ( z[1-α] + z[1-β])^2 / (1/2 ln{(1+q)/(1-q)}) ^2 ) + 3
The [1-α] and [1-β] means subscript, t.i. qauantiles. ^2 means square
power.
The q means expected correlation coefficent.
This formula could be find in "Machin D, Campbell MJ, Fayers PM, Pinol
APY. Sample size tables for clinical studies. Oxford: Blackwell
Science; 1997."
My friend suggest to select smaller q for calculations, and if the
study shows that "real" q is greater then predicted, it does nothing
bad, I mean, the power will not be lost.
Maybe it is the right solution for me?
Thanks in advance.
A.B.
beisikas
Jul 15, 2011, 4:46:33 AM7/15/11
to MedStats
Sorryr, the formula was sent uncorrectly.
N= ( z[1-alfa] + z[1-beta])^2 / (1/2 ln{(1+q)/(1-q)}) ^2 ) + 3
Alfa and beta means appopriate greek letters.
N= ( z[1-alfa] + z[1-beta])^2 / (1/2 ln{(1+q)/(1-q)}) ^2 ) + 3
Alfa and beta means appopriate greek letters.
beisikas
Jul 16, 2011, 10:19:23 AM7/16/11
to MedStats
Anyone, please :)
Thanks in advance.
Thanks in advance.
MartinHolt
Jul 17, 2011, 8:36:20 AM7/17/11
to MedStats
Hi Andrius,
No-one's commented on this discussion for a while yet, and I've been
trying to get my head around how you can achieve your aims stated July
15th, 9.31am above. Maybe what follows will at the least prompt others
to comment on how to go.....sorry if I'm wide of the mark.
Steve Simon was the first to reply and said, "You will get a lot of
calculating sample size based on confidence intervals for sensitivity
and specificity (and later I provided a link that took that approach).
Mixed up in this is that the sensitivity/specificity approach that I
suggested is based on X-values and Y-values (not: diseased or not
diseased) with you experimenting with cut-offs for X-values and Y-
values.
Steve Simon said in his first response "By the way, you need data on
was based on just X-values and Y-values, without looking at whether
participants were healthy or not.
Whatever, I think the key response is Frank Harrell's (which I think
runs counter to Steve's and my suggestions): "In addition, I can see
regression) approach that I suggested, which would I think address
your first aim. If you perform a functional relationship, if the
confidence interval for the intercept includes zero , then the
intercept is not statistically different to zero. Likewise. if the
confidence interval of the slope includes 1.00, the slope is not
significantly different to the ideal (Wild et al). So the crunch
question is, "How many participants are required to run a functional
relationship?" Wild et al recommend at least 100. In a previous life I
worked with the authors of Wild et al ("The Immunoassay Handbook"
edited by David Wild, 1994). If you're interested, I don't know if
it's still in print, but I'd bet that you could get a copy from
Abebooks.)
Maybe Frank could comment? or expand on his comment above? I like
Neil's signature: "“Truth in science can be defined as the working
Best Wishes,
Martin
No-one's commented on this discussion for a while yet, and I've been
trying to get my head around how you can achieve your aims stated July
15th, 9.31am above. Maybe what follows will at the least prompt others
to comment on how to go.....sorry if I'm wide of the mark.
Steve Simon was the first to reply and said, "You will get a lot of
comments along the lines of "you shouldn't analyze
it this way because..." and I'd encourage you to look at those
comments carefully and with an open mind." He went on to suggest
it this way because..." and I'd encourage you to look at those
calculating sample size based on confidence intervals for sensitivity
and specificity (and later I provided a link that took that approach).
Mixed up in this is that the sensitivity/specificity approach that I
suggested is based on X-values and Y-values (not: diseased or not
diseased) with you experimenting with cut-offs for X-values and Y-
values.
Steve Simon said in his first response "By the way, you need data on
men with disease in order to compute sensitivity OR to establish a
reasonable cutoff OR to compute an ROC curve. Do you really plan to
study ONLY healthy men (I assume that "healthy me" is a typo and not
an indication of self-experimentation). If so, then you need to
radically revise your statistical approach." The approach I suggested
reasonable cutoff OR to compute an ROC curve. Do you really plan to
study ONLY healthy men (I assume that "healthy me" is a typo and not
an indication of self-experimentation). If so, then you need to
was based on just X-values and Y-values, without looking at whether
participants were healthy or not.
Whatever, I think the key response is Frank Harrell's (which I think
runs counter to Steve's and my suggestions): "In addition, I can see
no motivation for seeking cutoffs or using sensitivity or specificity
in this context. In addition, it is highly unlikely that a cutoff
even exists. The relationship is much more likely to be smooth than
to be discontinuous."
(I think this supports the functional relationship (Deming's
in this context. In addition, it is highly unlikely that a cutoff
even exists. The relationship is much more likely to be smooth than
to be discontinuous."
regression) approach that I suggested, which would I think address
your first aim. If you perform a functional relationship, if the
confidence interval for the intercept includes zero , then the
intercept is not statistically different to zero. Likewise. if the
confidence interval of the slope includes 1.00, the slope is not
significantly different to the ideal (Wild et al). So the crunch
question is, "How many participants are required to run a functional
relationship?" Wild et al recommend at least 100. In a previous life I
worked with the authors of Wild et al ("The Immunoassay Handbook"
edited by David Wild, 1994). If you're interested, I don't know if
it's still in print, but I'd bet that you could get a copy from
Abebooks.)
Maybe Frank could comment? or expand on his comment above? I like
Neil's signature: "“Truth in science can be defined as the working
hypothesis best suited to open the way to the next better one.” -
Konrad Lorenz.
Best Wishes,
Martin
Frank Harrell
Jul 17, 2011, 9:55:51 AM7/17/11
to MedStats
Just a few random comments.
Sensitivity and specificity may be useful if doing a case-control
study and the outcome Y is truly all-or-nothing.
If the underlying Y truly is all-or-nothing (and in general such a Y
does not exist), estimating the probability that Y=1 may be the goal.
The first consideration of sample size then could be achieving a good
margin of error in estimating the intercept. In the worst case (true
intercept = 0), the sample size required to achieve a +/- 0.1 margin
of error with 0.95 confidence is n=96. If there was a single
predictor, consisting of k mutually exclusive categories with equal
sample sizes, 96k observations would be needed to achieve a margin of
error of +/- 0.1 for all levels of X in predicting Prob(Y=1).
Cutoffs often create gaming (witness income tax brackets and fudging
data to qualify patients for clinical trials on the basis of
continuous patient characteristics). They always create
heterogeneity.
Frank
Sensitivity and specificity may be useful if doing a case-control
study and the outcome Y is truly all-or-nothing.
If the underlying Y truly is all-or-nothing (and in general such a Y
does not exist), estimating the probability that Y=1 may be the goal.
The first consideration of sample size then could be achieving a good
margin of error in estimating the intercept. In the worst case (true
intercept = 0), the sample size required to achieve a +/- 0.1 margin
of error with 0.95 confidence is n=96. If there was a single
predictor, consisting of k mutually exclusive categories with equal
sample sizes, 96k observations would be needed to achieve a margin of
error of +/- 0.1 for all levels of X in predicting Prob(Y=1).
Cutoffs often create gaming (witness income tax brackets and fudging
data to qualify patients for clinical trials on the basis of
continuous patient characteristics). They always create
heterogeneity.
Frank
beisikas
Jul 18, 2011, 11:42:37 AM7/18/11
to MedStats
Big big thanks to Mr. Frank and Mr. Martin ! :)
I have also read somewhere that for studies on seeking relationships
sample size should be not less then 100.
And the last question - about the formula provided by me on 15th July,
based on correliaton coefficient. I think it is also wouldn't be the
worst idea?
Regards,
Andrius
I have also read somewhere that for studies on seeking relationships
sample size should be not less then 100.
And the last question - about the formula provided by me on 15th July,
based on correliaton coefficient. I think it is also wouldn't be the
worst idea?
Regards,
Andrius
Reply all
Reply to author
Forward
0 new messages