Statistical collaboration
John B
We are trying to contact someone interesting to help us in a grant
application (clinical research involving seom sample size calculation
at the beginning, and time-series and ROC curves analsyis during the
project). I do not know if this is the right place to post. such a
request. Could we do it or alternatively could you please let us now
where to post such a request ?
Thanks
ציפי שוחט
Tzippy Shochat
--
To post a new thread to MedStats, send email to MedS...@googlegroups.com .
MedStats' home page is http://groups.google.com/group/MedStats .
Rules: http://groups.google.com/group/MedStats/web/medstats-rules
Frank Harrell
of false positive and false negative are dictated by data instead of
humans.
Frank
Karl Schlag
Could it be that you are implicitly criticising ROC analysis. If, so
could you spell out your arguments?
Or point me/us to references.
Thanx, Karl
(ps: started a new thread as this only indirectly touches on the
original query)
--
---------------------------------------------------------------------
Karl Schlag
Professor Tel: +43-1-4277-374-37
Department of Economics Fax: +43-1-4277-9374
University of Vienna email: karl....@univie.ac.at
Hohenstaufengasse 9 Room: 505
1010 Vienna, Austria
Pedro Emmanuel Alvarenga Americano do Brasil
Abraço forte e que a força esteja com você,
Dr. Pedro Emmanuel A. A. do Brasil
Instituto de Pesquisa Clínica Evandro Chagas
Fundação Oswaldo Cruz
Rio de Janeiro - Brasil
Av. Brasil 4365
Tel 55 21 3865-9648
email: pedro....@ipec.fiocruz.br
email: emmanue...@gmail.com
---Apoio aos softwares livres
www.zotero.org - gerenciamento de referências bibliográficas.
www.broffice.org ou www.openoffice.org - textos, planilhas ou apresentações.
www.epidata.dk - entrada de dados.
www.r-project.org - análise de dados.
www.ubuntu.com - sistema operacional
Pedro Emmanuel Alvarenga Americano do Brasil
Abraço forte e que a força esteja com você,
Dr. Pedro Emmanuel A. A. do Brasil
Instituto de Pesquisa Clínica Evandro Chagas
Fundação Oswaldo Cruz
Rio de Janeiro - Brasil
Av. Brasil 4365
Tel 55 21 3865-9648
email: pedro....@ipec.fiocruz.br
email: emmanue...@gmail.com
---Apoio aos softwares livres
www.zotero.org - gerenciamento de referências bibliográficas.
www.broffice.org ou www.openoffice.org - textos, planilhas ou apresentações.
www.epidata.dk - entrada de dados.
www.r-project.org - análise de dados.
www.ubuntu.com - sistema operacional
John B
On Mar 12, 11:11 am, ציפי שוחט <tz.shoc...@gmail.com> wrote:
> Try LinkedIn - Global Statistics Networking Association.
> Tzippy Shochat
>
Frank Harrell
majority of cases we are interested in making optimal decisions for
individuals. ROC analysis is more for group decision making, and many
patients resent being treated like other patients because someone
arbitrarily grouped them together. Some good references are below. -
Frank
@Article{bri08ski,
author = {Briggs, William M. and Zaretzki, Russell},
title = {The skill plot: {A} graphical technique for evaluating
continuous diagnostic tests (with discussion)},
journal = Biometrics,
year = 2008,
volume = 64,
pages = {250-261},
annote = {ROC curve;sensitivity;skill plot;skill
score;specificity;diagnostic accuracy;diagnosis;``statistics such as
the AUC are not especially relevant to someone who must make a
decision about a particular $x_{c}$. \ldots ROC curves lack or
obscure several quantities that are necessary for evaluating the
operational effectiveness of diagnostic tests. \ldots ROC curves were
first used to check how radio \emph{receivers} (like radar receivers)
operated over a range of frequencies. \ldots This is not how must ROC
curves are used now, particularly in medicine. The receiver of a
diagnostic measurement \ldots wants to make a decision based on some
$x_{c}$, and is not especially interested in how well he would have
done had he used some different cutoff.''; in the discussion David
Hand states ``when integrating to yield the overall AUC measure, it is
necessary to decide what weight to give each value in the
integration. The AUC implicitly does this using a weighting derived
empirically from the data. This is nonsensical. The relative
importance of misclassifying a case as a noncase, compared to the
reverse, cannot come from the data itself. It must come externally,
from considerations of the severity one attaches to the different
kinds of misclassifications.''; see Lin, Kvam, Lu Stat in Med
28:798-813;2009}
}
@Article{han10eva,
author = {Hand, David J.},
title = {Evaluating diagnostic tests: {The} area under the {ROC}
curve and the balance of errors},
journal = Stat in Med,
year = 2010,
volume = 29,
pages = {1502-1510},
annote = {diagnosis;diagnostic accuracy;fundamental problems with
ROC area due to failure to balance difference kinds of misdiagnoses
effectively;proposal to use H index discussed in Hand DJ:Machine
Learning 77:103-123;2009}
roland andersson
Making a diagnosis is not black or white. Different test can give
different information even if ROC area is the same. However, I give
the test with largest ROC area the most attention. How can you tell
what test to use for an individual? I think this is more a question
when it comes to the interpretation of the test in view of the while
clinical picture. You should use the test with the largest ROC area,
and be aware that most test are imperfect.
Roland Andersson
2011/3/13 Frank Harrell <f.ha...@vanderbilt.edu>:
> --
John B
We are intereste sin this type of approach to ROC analysis applied to
Biomarkers and for instance outcome (Good or bad). I copy a recent
methodology:
Receiver-operating characteristic (ROC) curves were used to determine
the optimal cut-off point for biochemical markers in predicting poor
outcome. A curve was
obtained by plotting sensitivity against the false-positive rate (1
minus specificity) for all possible cut-off points of the biochemical
markers. The area under the ROC curve was used as an index of the
discriminating ability of the biochemical markers . This area has
values ranging from zero to one: 0.5 implies a test with an ability to
discriminate no better than chance, while 1.0 indicates a test with
perfect positive discrimination; areas greater than 0.7 are generally
thought to be useful. The optimal cut-off value above which
biochemical markers would provide the greatest yield was selected by
choosing the point that lay furthest from the 45. line. Diagnostic
accuracy of the biochemical markers was assessed by estimating their
sensitivity, specificity, positive and negative predictive values,
accuracy, and 95% confidence intervals.
Do you believe this is a god approach?
roland andersson
I guess two cutoff points with high sensitivity and specificity and 3
testzones (disease very likely, indeterminate and very unlikely) will
work better in the clinical situation.
Roland Andersson
2011/3/13 John B <bangali...@gmail.com>:
Frank Harrell
patient depend on how you diagnosed his next-door neighbor? To put it
another way, a good clinician will make use of the fact that a patient
has a systolic blood pressure of 158mmHg and will not be interested
only in whether the patient's blood pressure exceeds 140mmHg. There
is little sense in conditioning on X > c when you can condition on X =
x. Fortunately, physicians invite only one patient at a time into the
examining room and do not need to make group decisions. The binary
logistic regression model is your friend. An ordinal logistic
regression model (recognizing that diagnoses are usually not binary)
is even more useful.
A somewhat silly analogy may help.
The Dichotomizing Motorist
--------------------------
The speed limit is 60.
I am going faster than the speed limit.
Will I be caught?
An answer by a dichotomizer:
Are you going faster than 70?
An answer from a better dichotomizer:
If you are among other cars, are you going faster than 73?
If you are exposed are your going faster than 67?
Better:
How fast are you going and are you exposed?
Analogy to most medical diagnosis research in which +/- diagnosis
is a false dichotomy of an underlying disease severity:
The speed limit is moderately high.
I am going fairly fast.
Will I be caught?
Frank
Max Jasper
Actually, doc is not much interested in having an exact systolic bp to make decisions. All needed is to know that one's bp is higher than normal of something like 120...and guess what, 120 comes from the ROC!!! or in case of patients with diabetes, 130 comes from the ROC too! Of course what you say is only true in my case because I need my doc to give me my exact bp at each visit for my stat records!
Frank Harrell
uses your absolute blood pressure, measured to the accuracy possible
with the chosen device. Your doctor does not use the fact that your
blood pressure is less than or greater than a certain value. The
easiest way to see that is to see how your doctor weighs your
diastolic blood pressure, cholesterol, BMI, alcohol consumption, ...
when interpreting systolic blood pressure. Having more risk factors
present creates more concern over any one of the risk factors. Check
the Framingham risk model and its simplication as used in the National
Cholesterol Education Program. ROC analysis clouds rather than
illuminates, and creates a temptation for people to use improper
scoring rules (discontinuous proportions such as classification
accuracy, sensitivity, specificity) rather than proper scoring rules
(log likelihood, logarithmic scoring rule, Brier quadratic scoring
rule, etc.).
Frank
On Mar 13, 5:55 pm, Max Jasper <maxjas...@shaw.ca> wrote:
> Frank said: *"...To put it another way, a good clinician will make use of
> not be interested only in whether the patient's blood pressure exceeds
> 140mmHg..."
>
> something like 120...and guess what, 120 comes from the ROC!!! or in case of
> patients with diabetes, 130 comes from the ROC too! Of course what you say
> is only true in my case because I need my doc to give me my exact bp at each
> *
Max Jasper
roland andersson
I do not uderstand your point.
Your analogy works for bloodpressure but not for other biological
tests where the diagnosis is not black or white but there is an area
of indeterminate diagnosis. The definition of hypertension is like the
speed limit. Someone has decided where the limit is. If you plot an
ROC area for bloodpressure and hypertension you will get an area of
1.0. For appendicitis and WBC count or CRP it is quite different. The
area is about 0.70-0.80. You can not define one point that has both
high sensitivity and specificity.
When I construct a diagnostic test I need of course a large group of
neighbours with suspicion of the disease. How can I otherwise know the
association between a test and disease. When i meet a new patient why
should I not compare him with the nearest neighbours in order to get
some idea of the prognosis?
It is of course another story what I will do with this information. If
the diagnostic test tells me that 90% of neighbours with this
testresult has appendicitis , am I willing to take the risk of a
negative exploration? Can I get more information? But the information
I get from the test is stuill useful.
Greetings
Roland Andersson
2011/3/13 Frank Harrell <f.ha...@vanderbilt.edu>:
Frank Harrell
On Mar 14, 10:17 am, roland andersson <rolanders...@gmail.com> wrote:
> Frank
>
> I do not uderstand your point.
>
> Your analogy works for bloodpressure but not for other biological
> tests where the diagnosis is not black or white but there is an area
> of indeterminate diagnosis. The definition of hypertension is like the
> speed limit. Someone has decided where the limit is. If you plot an
> ROC area for bloodpressure and hypertension you will get an area of
> 1.0. For appendicitis and WBC count or CRP it is quite different. The
> area is about 0.70-0.80. You can not define one point that has both
> high sensitivity and specificity.
There is no single real limit for BP. And an ROC area would only be
1.0 with a circular definition. Tying BP to bad clinical outcomes
would yield an ROC area < 1.0.
>
> When I construct a diagnostic test I need of course a large group of
> neighbours with suspicion of the disease. How can I otherwise know the
> association between a test and disease. When i meet a new patient why
> should I not compare him with the nearest neighbours in order to get
> some idea of the prognosis?
The risk estimate you make for a single patient is conditional on only
that patient's set of covariate values. You don't use neighbors at
this final estimation step.
>
> It is of course another story what I will do with this information. If
> the diagnostic test tells me that 90% of neighbours with this
> testresult has appendicitis , am I willing to take the risk of a
> negative exploration? Can I get more information? But the information
> I get from the test is stuill useful.
need to know about baseline covariates. A probability is self-
contained, i.e., contains its own error measure. If you don't treat a
patient who has a risk of 0.1 then you are by definition making a
mistake 10% of the time. This is why you don't want to discard any
information (e.g., only record risk < 0.10 when the risk is 0.01) that
would help with the decision.
Frank
>
> Greetings
>
> Roland Andersson
>
> 2011/3/13 Frank Harrell <f.harr...@vanderbilt.edu>:
John Whittington
> There is no single real limit for BP. And an ROC area would only be
>1.0 with a circular definition. Tying BP to bad clinical outcomes
>would yield an ROC area < 1.0.
I think Roland's point is probably that certain diseases/abnormal states
(e.g. hypertension, diabetes) are already defined in terms of
internationally-agreed (and regularly reviewed) fixed threshold
measurements - which are hopefully based on risk analyses (i.e.
relationship between measurements of the quantity concerned and 'bad
clinical outcomes') deriving from very large sets of data.
In situations like this, I think it would be confusing and arguably
'meddlesome' to attempt to re-invent that wheel and interpret, say, blood
pressure and blood glucose levels in relation to one's own view of the risk
analysis. That could result in some people being diagnosed as having the
diseases in question despite not fulfilling the internationally-agreed
criteria, or vice versa - and, whilst the scientific justification for such
an approach may be sound (depending on the size and scope of the data set
one used as a basis for one's risk assessment, it certainly would be a
confusing situation, for all concerned.
That's how I see it, anyway!
Kind Regards,
John
----------------------------------------------------------------
Dr John Whittington, Voice: +44 (0) 1296 730225
Mediscience Services Fax: +44 (0) 1296 738893
Twyford Manor, Twyford, E-mail: Joh...@mediscience.co.uk
Buckingham MK18 4EL, UK
----------------------------------------------------------------
Thompson,Paul
You state "If you don't treat a patient with a risk of .1, you by definition make a mistake 10% of the time". I know that you are not a clinician, nor am I. I submit that your answer is pretty useless to a clinician, however.
You need to either treat or not treat. You need to minimize risk of false treatment (thus medicating inappropriately) and false lack of treatment (thus incurring losses of death and morbidity). Certain treatments can be titrated, but many cannot. If you prescribe antibiotics, titration is an unacceptable choice.
Thus, a treatment must be given when appropriate. It is a binary choice in most cases. Even in cases in which levels of treatment can be administered, each must be prescribed or not. Under that view, why are ROC approaches to diagnostic performance incorrect?
Paul A. Thompson, Ph.D.
Additional contact numbers:
Cell: 618-974-0473
Fax: 605-312-6071
Rules: http://groups.google.com/group/MedStats/web/medstats-rules
Please note that My Email Address Has Changed!
Please begin using the address in the "From" line above, immediately.
Soon, email sent to my old address will no longer be delivered to me.
Confidentiality Notice: This e-mail message, including any attachments,
is for the sole use of the intended recipient(s) and may contain
privileged and confidential information. Any unauthorized review, use,
disclosure or distribution is prohibited. If you are not the intended
recipient, please contact the sender by reply e-mail and destroy
all copies of the original message.
Greg Snow
So if I understand your argument correctly, then anyone with a probability of disease of 51% should be treated exactly the same as someone with a 98% probability of disease? And the person with a 49% chance of disease should be not be treated even though his/her chance is only 2% less than the 1st person? And all of this should ignore any additional information that the doctor and patient have that was not included in the ROC analysis (family history, prior tests, current epidemics, ...). And should patient preferences play a part?
If I were the patient I would prefer that the doctor come to me and tell me what my probability of disease is, then we could discuss other factors that may influence that probability (I know that sounds a bit Bayesian, but for my treatment I think my subjectivity is relevant). We could also then discuss the effects of the treatment and the effects of not treating the disease, then together make a decision about my treatment (that whole informed consent issue). If the doctor only receives a +/- result rather than a probability, can any decision we make really be considered informed?
If a doctor/patient combination really only wants a +/- result, then they can choose a cutoff on the probability and easily convert to +/-, but it is much harder to convert a +/- back into a meaningful probability for those that want to make their decisions with as much information as possible.
--
Gregory (Greg) L. Snow Ph.D.
Statistical Data Center
Intermountain Healthcare
greg...@imail.org
801.408.8111
Frank Harrell
On Mar 14, 12:23 pm, John Whittington <Joh...@mediscience.co.uk>
wrote:
>
> > There is no single real limit for BP. And an ROC area would only be
> >1.0 with a circular definition. Tying BP to bad clinical outcomes
> >would yield an ROC area < 1.0.
>
> I think Roland's point is probably that certain diseases/abnormal states
> (e.g. hypertension, diabetes) are already defined in terms of
> internationally-agreed (and regularly reviewed) fixed threshold
> measurements - which are hopefully based on risk analyses (i.e.
> relationship between measurements of the quantity concerned and 'bad
> clinical outcomes') deriving from very large sets of data.
cases.
>
> In situations like this, I think it would be confusing and arguably
> 'meddlesome' to attempt to re-invent that wheel and interpret, say, blood
> pressure and blood glucose levels in relation to one's own view of the risk
> analysis. That could result in some people being diagnosed as having the
> diseases in question despite not fulfilling the internationally-agreed
> criteria, or vice versa - and, whilst the scientific justification for such
> an approach may be sound (depending on the size and scope of the data set
> one used as a basis for one's risk assessment, it certainly would be a
> confusing situation, for all concerned.
patients. Physicians do trade-offs all the time (do I tell her to
quit smoking or to reduce cholesterol?...) Google "Against Diagnosis"
for more.
Frank
Frank Harrell
What you stated is not consistent with what probabilities mean. Those
who bet on horse races or football games are quite good at
understanding and using probabilities.
Frank
On Mar 14, 12:24 pm, "Thompson,Paul" <Paul.Thomp...@SanfordHealth.org>
wrote:
John Whittington
>What you stated is not consistent with what probabilities mean. Those
>who bet on horse races or football games are quite good at
>understanding and using probabilities.
As I said before, I really don't understand these betting analogies. Most
punters gamble on the fact that they will 'be lucky' and make a lot of
money, DESPITE the low probability of a long-odds result occurring. The
nearest clinical analogy to that I can think of would be to reject a
treatment that was known to have a high probability of achieving moderate
success in favour of one which had a very low probability of working, but
which worked extremely well in the very few cases in which it did work!
Frank Harrell
against my $1 if the odds of your winning are 2:1 or prob. is 2/3)
explains a lot in my opinion. Probability is something that applies
to the moment. That's why baseball managers look at left and right-
field hitting percentages of batters when aligning outfielders. You
go with the odds, i.e., long-range tendencies for deciding what to do
in an individual case. That's the best we can do. If you know the
cost of false positive and false negative then formal decision
analysis tells us how to plug in the predicted risk to make an optimum
decision.
Frank
John Whittington
>On Mar 14, 12:23 pm, John Whittington <Joh...@mediscience.co.uk>
>wrote:
> > I think Roland's point is probably that certain diseases/abnormal states
> > (e.g. hypertension, diabetes) are already defined in terms of
> > internationally-agreed (and regularly reviewed) fixed threshold
> > measurements - which are hopefully based on risk analyses (i.e.
> > relationship between measurements of the quantity concerned and 'bad
> > clinical outcomes') deriving from very large sets of data.
>
>cases.
I really think that most are - albeit necessarily on a population, rather
than individual, basis.
>Formal risk analysis will result in decidedly better decisions for
>patients. Physicians do trade-offs all the time (do I tell her to
>quit smoking or to reduce cholesterol?...) Google "Against Diagnosis"
>for more.
You certainly don't need to tell me that physicians "do trade-offs all the
time"; I spent many tears doing such 'trading off' :-) As I implied
before, scientifically/statistically speaking, you are probably right in
suggesting that risk analysis on a 'per-patient' basis is best for the
patient - and, per the 'trading off', a lot of that is necessarily
happening all the time, whether formally or informally. However, I think
you may underestimate the practical problems (for both patients and
healthcare professionals) that would result from losing universal fixed
definitions of diseases.
I think there may be some confusion here between diagnosis and
treatment. I really do think that practicalities more-or-less require that
we have fixed definitions of things like diabetes (for example, in the UK,
that diagnosis has the effect of making _all_ medication required by the
patient for the rest of their life free-of-charge). However, having
established a diagnosis, whether and how one treats the patient is a
totally different matter, and many other factors go into that melting pot
(essentially a 'risk analysis', even if 'informal').
roland andersson
I am happy that doctors do not count like baseball managers. The
baseball manager expect to win the league. For him a couple of losses
do not count. We expect to win each run and each game.
Statistics is a good help, but is quite different from clinical practice.
Greetings
Roland Andersson
2011/3/14 Frank Harrell <f.ha...@vanderbilt.edu>:
John Whittington
>di Finetti's views of probability in terms of a fair bet (you bet $2
>against my $1 if the odds of your winning are 2:1 or prob. is 2/3)
>explains a lot in my opinion. Probability is something that applies
>to the moment. That's why baseball managers look at left and right-
>field hitting percentages of batters when aligning outfielders. You
>go with the odds, i.e., long-range tendencies for deciding what to do
>in an individual case. That's the best we can do. If you know the
>cost of false positive and false negative then formal decision
>analysis tells us how to plug in the predicted risk to make an optimum
>decision.
I don't disagree with any of that. "Going with the odds" is a rational use
of probabilities (or odds!!).
However, that still doesn't help me to understand the appropriateness of
the 'betting' analogies you keep using. Most (non-'professional') punters
effectivcely go _against_ the odds, gambling that they will be blessed by
the 'improbable' happening, to the benefit of their pocket! If the odds are
set appropriately, it can be a 'fair bet' (although the bookmaker will
obviously want an income, which almost certainly makes the bet 'unfair'),
but one which one is unlikely to win.
Whatever, one does not gamble unnecessarily with patients' health and lives
- one will make the best 'rational' use of the available probabilistic
information.
Frank Harrell
On Mar 14, 3:53 pm, John Whittington <Joh...@mediscience.co.uk> wrote:
> At 13:01 14/03/2011 -0700, Frank Harrell wrote:
>
> >On Mar 14, 12:23 pm, John Whittington <Joh...@mediscience.co.uk>
> >wrote:
> > > I think Roland's point is probably that certain diseases/abnormal states
> > > (e.g. hypertension, diabetes) are already defined in terms of
> > > internationally-agreed (and regularly reviewed) fixed threshold
> > > measurements - which are hopefully based on risk analyses (i.e.
> > > relationship between measurements of the quantity concerned and 'bad
> > > clinical outcomes') deriving from very large sets of data.
>
> >Unfortunately they are not based on formal risk analysis in most
> >cases.
>
> I really think that most are - albeit necessarily on a population, rather
> than individual, basis.
that formal risk estimation was not used in most cases.
>
> >Formal risk analysis will result in decidedly better decisions for
> >patients. Physicians do trade-offs all the time (do I tell her to
> >quit smoking or to reduce cholesterol?...) Google "Against Diagnosis"
> >for more.
>
> You certainly don't need to tell me that physicians "do trade-offs all the
> time"; I spent many tears doing such 'trading off' :-) As I implied
> before, scientifically/statistically speaking, you are probably right in
> suggesting that risk analysis on a 'per-patient' basis is best for the
> patient - and, per the 'trading off', a lot of that is necessarily
> happening all the time, whether formally or informally. However, I think
> you may underestimate the practical problems (for both patients and
> healthcare professionals) that would result from losing universal fixed
> definitions of diseases.
>
> I think there may be some confusion here between diagnosis and
> treatment. I really do think that practicalities more-or-less require that
> we have fixed definitions of things like diabetes (for example, in the UK,
> that diagnosis has the effect of making _all_ medication required by the
> patient for the rest of their life free-of-charge). However, having
> established a diagnosis, whether and how one treats the patient is a
> totally different matter, and many other factors go into that melting pot
> (essentially a 'risk analysis', even if 'informal').
labels, don't serve us well. Think of the men treated for "mild"
prostate cancer who are going to die of something else.
Best,
Frank
John Whittington
> >......... However, I think
> > you may underestimate the practical problems (for both patients and
> > healthcare professionals) that would result from losing universal fixed
> > definitions of diseases.
>
Is that a typo, Frank? I was talking about _losing_, not _using_, fixed
definitions. Those fixed definitions of disease are with us, for better or
for worse, and their loss would, IMO, produce all sorts of
problems. Indeed, an awful lot of health-related statistics (at least, as
they currently are) would be undermined by non-consistent definitions.
> > I think there may be some confusion here between diagnosis and
> > treatment. I really do think that practicalities more-or-less require that
> > we have fixed definitions of things like diabetes .... However, having
> > established a diagnosis, whether and how one treats the patient is a
> > totally different matter, and many other factors go into that melting pot
> > (essentially a 'risk analysis', even if 'informal').
>
>I'll have to gently disagree with this. Binary diagnoses, like race
>labels, don't serve us well. Think of the men treated for "mild"
>prostate cancer who are going to die of something else.
I can but refer you to my words which you quoted - diagnosis and treatment
are different issues. However, if such is your point, whilst I think that
having a binary _qualitative_ diagnosis (even if the criteria are sometimes
verging on the arbitrary) has practical value, there is no reason for
diagnosis to be _quantitatively_ binary - i.e. it is perfectly reasonable
to subdivide a diagnosis according to, say, severity (e.g. your 'mild
prostate cancer'). ... and then, as I said before, whether/how/when to
treat (once a diagnosis has been made) is a totally separate question. As
you imply, many cases of 'mild prostate cancer' may not actually 'require'
treatment - but that does not alter the fact that they are cases of
prostatic cancer.
Just to be clear, if I put on my statistician's hat and forget all others,
I have considerable sympathy with everything you're saying - but if I reach
for my clinician's hat, or think of some of the practical, logistical and
administrative issues, then things change a bit. A few of those latter
issues are:
1...as already mentioned, in the UK the eligibility for free medications.
2...In relation to 'critical illness insurance' policies
3...In relation to healthcare insurance / travel insurance etc. policies,
which will often include or exclude treatment for certain diagnoses.
4...The 'on-label' use of licensed medications usually requires that it is
used to treat patients in whom a specific disease has been diagnosed.
5...In some cases there may be psychological and/or sociological impact
associated with certain diagnoses (e.g. if they are considered by some to
be 'stigmatising')
... and I'm sure there are many others. As things stand, all of those
areas required fixed (binary) definitions of diseases in order to be
workable, even those I agree that it is scientifically unpleasing that some
of those definitions are necessarily fairly arbitrary. I'm not saying that
couldn't be changed, but the amount of administrative and legal time/costs
involved would be very large indeed.
Frank Harrell
On Mar 15, 7:14 am, John Whittington <Joh...@mediscience.co.uk> wrote:
> At 19:05 14/03/2011 -0700, Frank Harrell wrote (in part):
>
> > >......... However, I think
> > > you may underestimate the practical problems (for both patients and
> > > healthcare professionals) that would result from losing universal fixed
> > > definitions of diseases.
>
> >I didn't suggest using universal fixed defs. of diseases.
>
> Is that a typo, Frank? I was talking about _losing_, not _using_, fixed
> definitions. Those fixed definitions of disease are with us, for better or
> for worse, and their loss would, IMO, produce all sorts of
> problems. Indeed, an awful lot of health-related statistics (at least, as
> they currently are) would be undermined by non-consistent definitions.
There are a lot of things that are "with us" that are mirages -
arbitrary conventions not tied to data. Think of apparent decreases
in cancer mortality that are due just to early diagnosis when tumors
are smaller. Think of diabetes - why does a patient with a gradually
increasing fasting blood glucose not have diabetes one day and have it
the next? Cognitive psychologists have shown us that dichotomization
is what we do when we are trying to process too many things at once.
It's an attempt to simplify, as in racial discrimination.
>
> > > I think there may be some confusion here between diagnosis and
> > > treatment. I really do think that practicalities more-or-less require that
> > > we have fixed definitions of things like diabetes .... However, having
> > > established a diagnosis, whether and how one treats the patient is a
> > > totally different matter, and many other factors go into that melting pot
> > > (essentially a 'risk analysis', even if 'informal').
>
> >I'll have to gently disagree with this. Binary diagnoses, like race
> >labels, don't serve us well. Think of the men treated for "mild"
> >prostate cancer who are going to die of something else.
>
> I can but refer you to my words which you quoted - diagnosis and treatment
> are different issues. However, if such is your point, whilst I think that
> having a binary _qualitative_ diagnosis (even if the criteria are sometimes
> verging on the arbitrary) has practical value, there is no reason for
> diagnosis to be _quantitatively_ binary - i.e. it is perfectly reasonable
> to subdivide a diagnosis according to, say, severity (e.g. your 'mild
> prostate cancer'). ... and then, as I said before, whether/how/when to
> treat (once a diagnosis has been made) is a totally separate question. As
> you imply, many cases of 'mild prostate cancer' may not actually 'require'
> treatment - but that does not alter the fact that they are cases of
> prostatic cancer.
severity (tumor volume, cell counts, glucose, H1c) is even better.
>
> Just to be clear, if I put on my statistician's hat and forget all others,
> I have considerable sympathy with everything you're saying - but if I reach
> for my clinician's hat, or think of some of the practical, logistical and
> administrative issues, then things change a bit. A few of those latter
> issues are:
>
> 1...as already mentioned, in the UK the eligibility for free medications.
around the (somewhat arbitrary) boundaries of qualifying conditions.
> 2...In relation to 'critical illness insurance' policies
> 3...In relation to healthcare insurance / travel insurance etc. policies,
> which will often include or exclude treatment for certain diagnoses.
> 4...The 'on-label' use of licensed medications usually requires that it is
> used to treat patients in whom a specific disease has been diagnosed.
> 5...In some cases there may be psychological and/or sociological impact
> associated with certain diagnoses (e.g. if they are considered by some to
> be 'stigmatising')
> ... and I'm sure there are many others. As things stand, all of those
> areas required fixed (binary) definitions of diseases in order to be
> workable, even those I agree that it is scientifically unpleasing that some
> of those definitions are necessarily fairly arbitrary. I'm not saying that
> couldn't be changed, but the amount of administrative and legal time/costs
> involved would be very large indeed.
Diagnostic and Statistical Manual away from binary diagnoses
(recognized by many psychiatrists to be arbitrary) into ordinal
gradings.
Best regards,
Frank
Frank Harrell
Baseball is vastly different form clinical practice to be sure. But
the underlying probabilitistic framework is the same.
Frank
@ARTICLE{spi86,
author = {Spiegelhalter, D. J.},
year = 1986,
title = {Probabilistic prediction in patient management and clinical
trials},
pages = {421-433},
annote = {shrinkage; prediction, general; predictive accuracy;
independence
(idiot Bayes) model; nonparametric calibration curve; z-
test for calibration inaccuracy (implemented in Stata)}
On Mar 14, 4:42 pm, roland andersson <rolanders...@gmail.com> wrote:
> Frank
>
> I am happy that doctors do not count like baseball managers. The
> baseball manager expect to win the league. For him a couple of losses
> do not count. We expect to win each run and each game.
>
> Statistics is a good help, but is quite different from clinical practice.
>
> Greetings
> Roland Andersson
>
John Whittington
>There are a lot of things that are "with us" that are mirages -
>arbitrary conventions not tied to data. Think of apparent decreases
>in cancer mortality that are due just to early diagnosis when tumors
>are smaller.
Indeed.
>Think of diabetes - why does a patient with a gradually
>increasing fasting blood glucose not have diabetes one day and have it
>the next?
I obviously agree ... but, as I said, we are currently stuck (in the UK)
with an administrative situation which makes the difference between those
two days of considerable importance to a patient - the difference between
paying 'standard charges' for medications and having all medications free
for the rest of their lives. Yes, this could be changed, but that's not
going to happen any time soon.
In any event, any 'threshold' situation inevitably results in questionable
(almost arbitrary) situations around that threshold. From the clinical
viewpoint, you could just as easily have asked why a patient with a
gradually increasing fasting blood glucose level doesn't require treatment
one day, but does require treatment the next. Unlike the question of
'diagnosis', which can be considered philosophically as a continuum', the
qualitative decision as to whether or not to treat a patient (yes/no)
inevitable IS a binary one.
>Yes, severity gradings are a vast improvement. Quantitative disease
>severity (tumor volume, cell counts, glucose, H1c) is even better.
>.......
>There is a move afoot in some areas of psychiatry to change the
>Diagnostic and Statistical Manual away from binary diagnoses
>(recognized by many psychiatrists to be arbitrary) into ordinal
>gradings.
I agree that's a step in the right direction, and, as I said before, I have
no problem with that. However, when it comes to practical/administrative
issues such as I listed (e.g. whether or not to give free medications,
cover under a policy, 'legal' on-label use of a medication etc.), those
ordinal gradings inevitably have to be collapsed into a dichotomy (yes/no),
since it is a binary decision (e.g. to pay or not to pay) that has to be
made. Attempts at any sort of 'sliding scale' of, say, payments (for
medications, insurance payouts etc.) based on ordinal gradings rather than
a binary diagnosis would almost certainly prove administratively unworkable
- or, at least, not cost-effective.
As I recently wrote to Roland, when it comes to treatment/management
decisions, involving a multifactorial approach, I do not see any merit in
'collapsing' the continuous probabilities at all - whether into a
dichotomy, trichotomy or 'multichotomy' of ordinal gradings - all of which
waste information. So I think we are agreed there.
Frank Harrell
the risk factors and concomitant diseases a patient has we would not
use a binary classification on the one disease at issue.
>
> As I recently wrote to Roland, when it comes to treatment/management
> decisions, involving a multifactorial approach, I do not see any merit in
> 'collapsing' the continuous probabilities at all - whether into a
> dichotomy, trichotomy or 'multichotomy' of ordinal gradings - all of which
> waste information. So I think we are agreed there.
Cheers
Frank
John Whittington
> > >Yes, severity gradings are a vast improvement. Quantitative disease
> > >severity (tumor volume, cell counts, glucose, H1c) is even better.
> > >.......
> > >There is a move afoot in some areas of psychiatry to change the
> > >Diagnostic and Statistical Manual away from binary diagnoses
> > >(recognized by many psychiatrists to be arbitrary) into ordinal
> > >gradings.
> >
> > I agree that's a step in the right direction, and, as I said before, I have
> > no problem with that. However, when it comes to practical/administrative
> > issues such as I listed (e.g. whether or not to give free medications,
> > cover under a policy, 'legal' on-label use of a medication etc.), those
> > ordinal gradings inevitably have to be collapsed into a dichotomy (yes/no),
> > since it is a binary decision (e.g. to pay or not to pay) that has to be
> > made. Attempts at any sort of 'sliding scale' of, say, payments (for
> > medications, insurance payouts etc.) based on ordinal gradings rather than
> > a binary diagnosis would almost certainly prove administratively unworkable
> > - or, at least, not cost-effective.
>
>the risk factors and concomitant diseases a patient has we would not
>use a binary classification on the one disease at issue.
Maybe I'm missing something, or perhaps you are talking about a utopia
which I would have thought would have been administratively so complex as
to be virtually impossible to implement. Maybe you can help me understand
your view by suggesting how you personally would handle a couple of the
issues I mentioned:
1...The decision by the UK NHS to provide free medication to people with
certain specified 'chronic diseases' (of which diabetes is one) is
obviously essentially a 'political' decision. Assuming that you didn't
want to attempt an even more complex system of having the NHS bear a
variable proportion of the cost of medication (depending on individual
patient circumstances), and that you were a politician who wanted, in some
way, to provide free medication to those with certain 'chronic illnesses',
what sort of decision rules would you apply to someone who we would today
say 'has diabetes' (per fixed diagnostic criteria) in order to determine
whether or not they qualified for free medications?
2...Similarly, if you were someone who wrote 'critical illness' insurance
policies, how would you word the bit that currently says that the policy
will pay out a certain sum if the patient is diagnosed as having, say, cancer?
Frank Harrell
wrote:
>
> > > >Yes, severity gradings are a vast improvement. Quantitative disease
> > > >severity (tumor volume, cell counts, glucose, H1c) is even better.
> > > >.......
> > > >There is a move afoot in some areas of psychiatry to change the
> > > >Diagnostic and Statistical Manual away from binary diagnoses
> > > >(recognized by many psychiatrists to be arbitrary) into ordinal
> > > >gradings.
>
> > > I agree that's a step in the right direction, and, as I said before, I have
> > > no problem with that. However, when it comes to practical/administrative
> > > issues such as I listed (e.g. whether or not to give free medications,
> > > cover under a policy, 'legal' on-label use of a medication etc.), those
> > > ordinal gradings inevitably have to be collapsed into a dichotomy (yes/no),
> > > since it is a binary decision (e.g. to pay or not to pay) that has to be
> > > made. Attempts at any sort of 'sliding scale' of, say, payments (for
> > > medications, insurance payouts etc.) based on ordinal gradings rather than
> > > a binary diagnosis would almost certainly prove administratively unworkable
> > > - or, at least, not cost-effective.
>
> >I'll have to disagree with this. If we really took into account all
> >the risk factors and concomitant diseases a patient has we would not
> >use a binary classification on the one disease at issue.
>
> Maybe I'm missing something, or perhaps you are talking about a utopia
> which I would have thought would have been administratively so complex as
> to be virtually impossible to implement. Maybe you can help me understand
> your view by suggesting how you personally would handle a couple of the
> issues I mentioned:
>
> 1...The decision by the UK NHS to provide free medication to people with
> certain specified 'chronic diseases' (of which diabetes is one) is
> obviously essentially a 'political' decision. Assuming that you didn't
> want to attempt an even more complex system of having the NHS bear a
> variable proportion of the cost of medication (depending on individual
> patient circumstances), and that you were a politician who wanted, in some
> way, to provide free medication to those with certain 'chronic illnesses',
> what sort of decision rules would you apply to someone who we would today
> say 'has diabetes' (per fixed diagnostic criteria) in order to determine
> whether or not they qualified for free medications?
feasibility. However, 'diabetes' is not one of them. The criteria
can and should be more specific. There is just too much heterogeneity
in outcomes for the label 'diabetes'.
>
> 2...Similarly, if you were someone who wrote 'critical illness' insurance
> policies, how would you word the bit that currently says that the policy
> will pay out a certain sum if the patient is diagnosed as having, say, cancer?
cancer should payout more than skin cancer.
Not easy questions though!
Frank
John Whittington
> > Maybe I'm missing something, or perhaps you are talking about a utopia
> > which I would have thought would have been administratively so complex as
> > to be virtually impossible to implement.
>
I didn't mean to imply that we don't have the knowledge/skill or IT
capabilities of implementing complex (non-binary) criteria. When I spoke
of the 'administrative complexity' in relation to the issues I was
discussing (eligibility for free medications, insurance policy payouts
etc.) I was thinking far more of the undoubted incredible volume of
disputes, appeals and litigation which would result from attempts to use
such criteria.
There is also the consideration that, with such a system, the vast majority
of the general public (not to mention many 'experts') probably would not be
able to even understand what criteria had to be met (to get them their free
medications or an insurance payout).
>I agree that certain things have to be simplified for the sake of
>feasibility. However, 'diabetes' is not one of them. The criteria
>can and should be more specific. There is just too much heterogeneity
>in outcomes for the label 'diabetes'.
I think you have to understand the political thinking/intention behind the
'free medication' thing in the UK. The idea is that people with 'certain
chronic illnesses' which are likely to lead to the need for life-long
medication should not have to pay for their medication - albeit that list
of 'certain chronic diseases' is a bit bizarre and much
criticized. However, one of them is "diabetes mellitus (except when
treated by diet alone)". It does not even distinguish between Type I and
Type II diabetes - but the implication is that it has to be _treated_
diabetes. For this purpose (and that's what I'm talking about) all one
therefore needs to know that there is a diagnosis of diabetes that requires
treatment.
> > 2...Similarly, if you were someone who wrote 'critical illness' insurance
> > policies, how would you word the bit that currently says that the policy
> > will pay out a certain sum if the patient is diagnosed as having, say,
> cancer?
>
>The best that might be done is to spell out 20 cancers. Surely lung
>cancer should payout more than skin cancer.
The sort of policy I'm talking about (maybe you don't have them 'over
there') does not have differential payouts. They simply pay out a
specified sum (the same in all cases) if the policyholder is diagnosed as
having a 'critical illness', as defined in the policy. That definition
usually amounts mainly to heart attacks, strokes, 'cancer' (with a few
exceptions) and neurodegenerative diseases. Relatively benign cancers like
basal cell carcinoma of the skin may be amongst the exceptions. However,
the trigger for a payout is _diagnosis_ of one of the specified (but not
excluded) diseases, which therefore needs to be, or be reducible to, a
binary criterion.
I suspect that most p[eople would be very hesitant to take out such a
policy of the criteria for a payout was the result of a multifactorial
algorithm, which they would be very unlikely to really understand.
>Not easy questions though!
Quite!
Frank Harrell
system.
>
> > > 2...Similarly, if you were someone who wrote 'critical illness' insurance
> > > policies, how would you word the bit that currently says that the policy
> > > will pay out a certain sum if the patient is diagnosed as having, say,
> > cancer?
>
> >The best that might be done is to spell out 20 cancers. Surely lung
> >cancer should payout more than skin cancer.
>
> The sort of policy I'm talking about (maybe you don't have them 'over
> there') does not have differential payouts. They simply pay out a
> specified sum (the same in all cases) if the policyholder is diagnosed as
> having a 'critical illness', as defined in the policy. That definition
> usually amounts mainly to heart attacks, strokes, 'cancer' (with a few
> exceptions) and neurodegenerative diseases. Relatively benign cancers like
> basal cell carcinoma of the skin may be amongst the exceptions. However,
> the trigger for a payout is _diagnosis_ of one of the specified (but not
> excluded) diseases, which therefore needs to be, or be reducible to, a
> binary criterion.
a certain way.
>
> I suspect that most p[eople would be very hesitant to take out such a
> policy of the criteria for a payout was the result of a multifactorial
> algorithm, which they would be very unlikely to really understand.
quantitative.
>
> >Not easy questions though!
>
> Quite!
Frank
John Whittington
that those getting bored with it can choose to ignore it!
At 19:26 15/03/2011 -0700, Frank Harrell wrote (in part):
> > ..... For this purpose (and that's what I'm talking about) all one
> > therefore needs to know that there is a diagnosis of diabetes that requires
> > treatment.
>
>I believe that binary classifications result in maximum gaming of the
>system.
In the situation we're talking about, it seems very unlikely that any
'gaming' would happen. Do you really believe that a physician would
conspire with a patient to institute treatment for diabetes
unnecessarily? Indeed, on the face of it, I would have thought that the
simpler a classification, the less scope there would be for it to be
manipulated.
One other issue that we haven't really discussed is the very serious
difficulty which would undoubtedly arise if one tempted to get a consensus
(which would really need to be global) of medical opinion on non-binary
diagnoses - there is enough argument already about criteria for binary
diagnoses.
Having said all that, there actually IS an internationally agreed (and to
some extent multifactorial) 'ordinal classification' for people with
elevated fasting blood glucose levels - and 'diabetes mellitus' is simply
the 'top' of the categories. Furthermore, in contrast to what you
suggested, it is very much a risk-based classification - see (50 pages, but
it does have a synopsis!):
http://www.who.int/diabetes/publications/Definition%20and%20diagnosis%20of%20diabetes_new.pdf
The point here, therefore, is that there IS a sophisticated and risk-based
classification of these disorders of glucose metabolism BUT a diagnosis of
'diabetes mellitus' itself remains 'binary' (criteria satisfied or not).
A similar thing happens in many other clinical fields - such that what you
are regarding as a 'binary diagnosis' is, in fact, merely one (usually the
'top') class from an ordinal, risk based and often multifactorial,
classification of related disorders. Take 'heart attacks' for
example. Myocardial infarction is myocardial infarction and is the binary
diagnosis we will most often be dealing with, However, 'below' that in the
classification are things like 'crescendo angina', 'unstable angina',
'coronary artery spasm' etc.
> > However, the trigger for a payout is _diagnosis_ of one of the specified
> > (but not excluded) diseases, which therefore needs to be, or be reducible
> > to, a binary criterion.
>
>I can see the pressures patients must put on physicians to code things
>a certain way.
To put this into perspective, in the vast majority of cases the diagnosis
is going to be totally clear-cut and hence neither contentious nor really
open to any 'negotiation', manipulation or pressure. There will obviously
always be some grey/gray areas (even between classifications in an ordinal
system), where even the physician is not sure what diagnosis to make, but
the insurers offer some guidance and also, needless to say, have review
processes in place for the less straightforward cases.
> > I suspect that most p[eople would be very hesitant to take out such a
> > policy of the criteria for a payout was the result of a multifactorial
> > algorithm, which they would be very unlikely to really understand.
>
>The algorithm doesn't have to be (necessarily) complex to be
>quantitative.
I fear that you probably overestimate a lot of the general public. I
believe that even the simplest of algorithms would be confusing, or not
understood, by many of them. I also think that you may be underestimating
the strength of the desire of very many patients to be given 'a diagnosis'
in simple language that they can understand, even if it represents an
oversimplification of the situation. Indeed, many patients seem to derive
some psychological comfort from 'being given a diagnosis' (even a very
serious one), even if it subsequently proves to have been incorrect.
Frank Harrell
On Mar 16, 5:52 am, John Whittington <Joh...@mediscience.co.uk> wrote:
> I thought I'd shift this tangential discussion into a thread of its own, so
> that those getting bored with it can choose to ignore it!
>
> At 19:26 15/03/2011 -0700, Frank Harrell wrote (in part):
>
> > > ..... For this purpose (and that's what I'm talking about) all one
> > > therefore needs to know that there is a diagnosis of diabetes that requires
> > > treatment.
>
> >I believe that binary classifications result in maximum gaming of the
> >system.
>
> In the situation we're talking about, it seems very unlikely that any
> 'gaming' would happen. Do you really believe that a physician would
> conspire with a patient to institute treatment for diabetes
> unnecessarily? Indeed, on the face of it, I would have thought that the
> simpler a classification, the less scope there would be for it to be
> manipulated.
income tax brackets and clinical trial entry criteria). Not so clear
in diabetes but I'd wager that there are patients being treated who
have less disease than some not being treated. Sometimes this is just
because of how wealthy is the patient.
>
> One other issue that we haven't really discussed is the very serious
> difficulty which would undoubtedly arise if one tempted to get a consensus
> (which would really need to be global) of medical opinion on non-binary
> diagnoses - there is enough argument already about criteria for binary
> diagnoses.
>
> Having said all that, there actually IS an internationally agreed (and to
> some extent multifactorial) 'ordinal classification' for people with
> elevated fasting blood glucose levels - and 'diabetes mellitus' is simply
> the 'top' of the categories. Furthermore, in contrast to what you
> suggested, it is very much a risk-based classification - see (50 pages, but
> it does have a synopsis!):
>
Cool
>
> The point here, therefore, is that there IS a sophisticated and risk-based
> classification of these disorders of glucose metabolism BUT a diagnosis of
> 'diabetes mellitus' itself remains 'binary' (criteria satisfied or not).
>
> A similar thing happens in many other clinical fields - such that what you
> are regarding as a 'binary diagnosis' is, in fact, merely one (usually the
> 'top') class from an ordinal, risk based and often multifactorial,
> classification of related disorders. Take 'heart attacks' for
> example. Myocardial infarction is myocardial infarction and is the binary
> diagnosis we will most often be dealing with, However, 'below' that in the
> classification are things like 'crescendo angina', 'unstable angina',
> 'coronary artery spasm' etc.
>
> > > However, the trigger for a payout is _diagnosis_ of one of the specified
> > > (but not excluded) diseases, which therefore needs to be, or be reducible
> > > to, a binary criterion.
clinical trial setting. A drug was ruled as ineffective just because
it reduced the number of serious MIs. Unfortunately for the sponsor
the main endpoint was all MIs (minor and major heart attacks combined)
and minor MIs washed out the treatment effect.
>
> >I can see the pressures patients must put on physicians to code things
> >a certain way.
>
> To put this into perspective, in the vast majority of cases the diagnosis
> is going to be totally clear-cut and hence neither contentious nor really
> open to any 'negotiation', manipulation or pressure. There will obviously
> always be some grey/gray areas (even between classifications in an ordinal
> system), where even the physician is not sure what diagnosis to make, but
> the insurers offer some guidance and also, needless to say, have review
> processes in place for the less straightforward cases.
>
> > > I suspect that most p[eople would be very hesitant to take out such a
> > > policy of the criteria for a payout was the result of a multifactorial
> > > algorithm, which they would be very unlikely to really understand.
>
> >The algorithm doesn't have to be (necessarily) complex to be
> >quantitative.
>
> I fear that you probably overestimate a lot of the general public. I
> believe that even the simplest of algorithms would be confusing, or not
> understood, by many of them. I also think that you may be underestimating
> the strength of the desire of very many patients to be given 'a diagnosis'
> in simple language that they can understand, even if it represents an
> oversimplification of the situation. Indeed, many patients seem to derive
> some psychological comfort from 'being given a diagnosis' (even a very
> serious one), even if it subsequently proves to have been incorrect.
Hundreds of thousands of patients have obtained "triple bypass
surgery" because of 3 non-significant coronary artery lesions that
caused their cardiologist to label the patient as having coronary
artery disease. Prostate cancer is an even better example.
Great dicussion; thanks for new topic heading. I hope this discussion
keeps returning to medstats. I think it is all-important, affecting
most of what we do in clinical biostatistics.
Frank
John Whittington
>Arbitrary thresholds result in gaming quite generally (take a look at
>income tax brackets and clinical trial entry criteria). Not so clear
>in diabetes but I'd wager that there are patients being treated who
>have less disease than some not being treated. Sometimes this is just
>because of how wealthy is the patient.
Well, thankfully not in the UK, for reason's I've explained!
However, the point surely is that 'patients being treated who have less
disease than some not being treated' can only really happen with _your_
approach. If the diagnosis were binary on the basis of fixed criteria, and
if having the diagnosis automatically led to treatment (so, again, binary)
(although that is not necessarily the case for Type II diabetes), then what
you are saying could not happen. It undoubtedly does happen that some
patients are treated who have less ('diabetes') disease than some who are
not treated, but when this happens it is due to the very thing you seem to
want to see - a multifactorial treatment decision in which factors other
than the (binary) diagnosis of diabetes are influencing the decision
whether or not to treat the diabetes. So, far from being undesirable
'gaming', what you describe sounds to me as if it's exactly what you like
to see!
>This has led to major problems. Here is one example, though in a
>clinical trial setting. A drug was ruled as ineffective just because
>it reduced the number of serious MIs. Unfortunately for the sponsor
>the main endpoint was all MIs (minor and major heart attacks combined)
>and minor MIs washed out the treatment effect.
Well, that's just a 'silliness' of unduly pedantic statistical/regulatory
thinking. Hopefully the sponsor went on to undertake a subsequent study
with a main endpoint suitable to demonstrate what the drug _could_ do.
From what you say, I would suspect that what may well have been happening
in that study was that the treatment was reducing the severity of serious
MIs , turning them into less serious ones, (minor heart attacks' as you put
it - although, as someone whose clinical years were mainly in/around
cardiology, I think I would question the concept of a 'minor heart attack',
not the least because the most 'trivial' {in terms of degree of coronary
artery blockage or myocardial damage} have the potential to be fatal!),
thereby leaving the total of minor+major relatively unchanged. Given that
such an occurrence might have been predicted as a possible outcome, I think
we're talking more of an imperfectly designed trial than anything directly
to do with the way that diagnoses are classified. In reality, the sponsor
may have considered that but, not knowing what the drug would do, thought
that it was probably a 'better bet' to go for all MIs!
...and, as I keep saying, I have absolutely no problem with subdivision (by
severity, or anything else appropriate) of 'binary' diagnoses - which is
what happens all ther time in clinical practice.
>This is a mirage that does not help the patient in the long run.
>Hundreds of thousands of patients have obtained "triple bypass
>surgery" because of 3 non-significant coronary artery lesions that
>caused their cardiologist to label the patient as having coronary
>artery disease.
That sounds like the US :-) Over here, resources for undertaking such
surgery (at least, by the NHS) is so restricted that only the most
deserving (high risk) cases are likley to get the surgery! Of course, we
also have some wealthy people, and some of them undoubtedly cajole the
private sector into performing surgery that would probably not be performed
on an NHS patient!
>Great dicussion; thanks for new topic heading. I hope this discussion
>keeps returning to medstats. I think it is all-important, affecting
>most of what we do in clinical biostatistics.
Agreed - but, from some of the things you've written, I think that you
probably underestimate the extent to which the approach you would like is
actually applied in clinical practice (certainly in the UK), even if
perhaps less formally than you would like.
Thompson,Paul
"I _knew_ you came from Afghanistan. From long habit the train of thoughts ran so swiftly through my mind, that I arrived at the conclusion without being conscious of intermediate steps. There were such steps, however. The train of reasoning ran, `Here is a gentleman of a medical type, but with the air of a military man. Clearly an army doctor, then. He has just come from the tropics, for his face is dark, and that is not the natural tint of his skin, for his wrists are fair. He has undergone hardship and sickness, as his haggard face says clearly. His left arm has been injured. He holds it in a stiff and unnatural manner. Where in the tropics could an English army doctor have seen much hardship and got his arm wounded? Clearly in Afghanistan.' The whole train of thought did not occupy a second. I then remarked that you came from Afghanistan, and you were astonished."
This is a quote from "A study in scarlet" and illustrates the manner in which "deduction" (a clearly incorrect term) is used by Holmes to apply a CATEGORICAL CLASSIFICATION to a series of CONTINUOUS OBSERVATIONS using a series of CONDITIONAL PROBABILITIES along with FILTERS and RESTRICTIONS. This is what happens, albeit less spectacularly, in many areas of diagnostic reasoning. Although every statement that Holmes makes is a continuous one, he comes to a strictly categorical conclusion. Thus also with medical diagnosis.
A child is presented with a) elevated temperature b) mottled skin c) indications from the parent that elevated temperature started 2 days ago, and others. Diagnosis: chicken pox. Confidence: 98%, especially if others from the same school were seen yesterday. Other diagnoses can be considered, but have a low likelihood (1%, 0%).
Pretty soon Frank is going to talk about black swans, I'm afraid.
Paul A. Thompson, Ph.D.
To post a new thread to MedStats, send email to MedS...@googlegroups.com .
MedStats' home page is http://groups.google.com/group/MedStats .
Rules: http://groups.google.com/group/MedStats/web/medstats-rules
John Whittington
>.....Here is a quote from a famous diagnostician:
>This is a quote from "A study in scarlet" and illustrates the manner in
>which "deduction" (a clearly incorrect term) is used by Holmes to apply a
>CATEGORICAL CLASSIFICATION to a series of CONTINUOUS OBSERVATIONS using a
>series of CONDITIONAL PROBABILITIES along with FILTERS and
>RESTRICTIONS. This is what happens, albeit less spectacularly, in many
>areas of diagnostic reasoning. Although every statement that Holmes makes
>is a continuous one, he comes to a strictly categorical conclusion. Thus
>also with medical diagnosis.
Exactly. In the final analysis, any decision can be reduced to a binary
one, or a series of binary ones (just as when we code dummy variables for a
regression analysis) - e.g. do we use dose 1 (Y/N)?, do we use dose 2
(Y/N)? ....etc. etc. - which is a formal way of stating the fact that,
ultimately, we either so something or don't do it (and, in the latter case,
might do something different instead).
I agree with the underlying spirit of a lot of what Frank has said, but I
think he is probably unnecessarily trying to constrain us into a formal
framework that is probably not necessary, and often not practical, but is
generally more-or-less being achieved by what already happens.
alejandro munoz
1 from the article Frank Harrell recently referred to (Vickers et al.
"Against Diagnosis" Ann Intern Med August 5, 2008 149:200-203). I
would add H Gilbert Welch's "Over-diagnosed. Making people sick in the
pursuit of health". Both sources distinguish between disease
(sickness, illness, disorder; anything that produces symptoms) and an
abnormality (conditions, risk factors).
Nobody denies that (binary) diagnosis is crucial for disease, or that
treatment is usually effective and necessary once such a diagnosis has
been determined. The problem is in trying to fit conditions into the
square hole of diagnosing a disease. By the definitions above, *in an
asymptomatic subject* high cholesterol, low bone density (T score),
high blood pressure, high body mass index, high PSA, etc. would all be
deemed conditions: there is nothing to treat (yet). Any symptoms or
bad outcome (fracture, cancer, renal failure, etc.) are only a
*future* *possibility*; hence the use of risk prediction models.
Establishing a cut-off or threshold to classify subjects into
"diseased" (osteoporotic, DM, obese, etc.) and "free of disease" and
treating them or not accordingly could be:
- arbitrary: why should someone with a fasting glucose of 123mg/dL be
any different than someone with 126mg/dL?;
- risky: side effects, drug interactions, post-op complications, etc.; and
- wasteful: some subjects will go on to experience the bad event
despite receiving treatment, and some treated subjects would have
never gone on to experience the bad event; similarly, some untreated
subjects may have benefitted from receiving treatment.
Welch's book documents fascinating examples of how ordinary, healthy
subjects all of a sudden become patients because a panel of experts
has decided that a threshold for disease should be moved.
If we keep these distinctions in mind I suspect we'd find that
apparent dissent is in fact agreement.
Alejandro
John Whittington
>Perhaps the problem is with terminology. I'd suggest consulting Table
>1 from the article Frank Harrell recently referred to (Vickers et al.
>"Against Diagnosis" Ann Intern Med August 5, 2008 149:200-203). I
>would add H Gilbert Welch's "Over-diagnosed. Making people sick in the
>pursuit of health". Both sources distinguish between disease
>(sickness, illness, disorder; anything that produces symptoms) and an
>abnormality (conditions, risk factors).
Indeed. You speak of that as if it were a new concept, but that
distinction was one of the first things I was taught (and repeatedly taught
thereafter0 when I entered medical school about 40 years ago. It is
closely related to "treat the patient, not the measurements/test
results/whatever", which was equally strongly drummed into us.
>....The problem is in trying to fit conditions into the
>square hole of diagnosing a disease. By the definitions above, *in an
>asymptomatic subject* high cholesterol, low bone density (T score),
>high blood pressure, high body mass index, high PSA, etc. would all be
>deemed conditions: there is nothing to treat (yet).
Agreed, except for elevated PSA - since it can, and often is, an indicator
that a disease (prostatic cancer) is already present.
>Any symptoms or
>bad outcome (fracture, cancer, renal failure, etc.) are only a
>*future* *possibility*; hence the use of risk prediction models.
Again, indeed - and that is what happens, to varying extents, in clinical
practice.
>Establishing a cut-off or threshold to classify subjects into
>"diseased" (osteoporotic, DM, obese, etc.) and "free of disease" and
>treating them or not accordingly could be:
>- arbitrary: why should someone with a fasting glucose of 123mg/dL be
>any different than someone with 126mg/dL?;
>- risky: side effects, drug interactions, post-op complications, etc.; and
>- wasteful: some subjects will go on to experience the bad event
>despite receiving treatment, and some treated subjects would have
>never gone on to experience the bad event; similarly, some untreated
>subjects may have benefitted from receiving treatment.
I don't think anyone is disputing any of that. 'Arbitrary' is, of course,
always going to be there, whatever processes one uses. As I keep saying,
the ultimate decision whether or not to treat is always inevitably a binary
one (yes/no) - which means there has to be a 'threshold'. Even if one
employs formal patient-specific risk prediction models, one ends up with a
probability, and threshold for treatment based on that probability ... so,
if that were the situation, I could equally ask you why, say, someone with
a predicted risk of a bad outcome of, say, 0.71 be any different from one
with a predicted risk of 0.70.
The 'risky' and 'wasteful' issues are things which are foremost in a
clinician's mind when making management decisions. Despite what Frank
seems to think, a diagnosis does not necessarily lead to a pre-defined
treatment, or necessarily any treatment at all.
Of course, many diagnoses (and the corresponding appropriate treatments)
really are very clear cut (albeit varying in severity), so these sort of
issues very often don't really arise.
>Welch's book documents fascinating examples of how ordinary, healthy
>subjects all of a sudden become patients because a panel of experts
>has decided that a threshold for disease should be moved.
Yes, I'm sure that happens. However, the same could happen if a panel
decided that to change the risk model/algorithm, or move the 'probability
threshold' to be used in turning its result into a clinical decision.
>If we keep these distinctions in mind I suspect we'd find that
>apparent dissent is in fact agreement.
I certainly do think that there is a lot more agreement than may appear to
be the case, but I think this is in great part due to Frank overlooking, or
being unaware of,' the extent to which the things he wants are already
happening, even if not as formally as he would like (and maybe sometimes
even 'subconsciously' on the part of the physician).
One factor which we have not touched on is the issue of litigation and
'defensive medicine', unfortunately somewhat led by those on the western
side of the Atlantic!! When, as is often the case, there are 'guidelines'
from national or international bodies indicating that a certain clinical
situation (maybe just 'a diagnosis') should be treated in a particular
fashion, clinicians will often be hesitant to deviate from that 'guidance',
even if they feel that it is probably not best/appropriate for the patient
in front of them, through fear that their deviation from this 'expert
guidance' could be thrown at them in a court of law, and might prevail in
the courts despite what they felt was a valid justification for their
different management. Sad, but increasingly true. We keep on coming back
to (Type II) diabetes, and this is probably a good example, since there are
some pretty specific 'internationally-agreed guidelines' as to which
patients should be treated, and how.
Frank Harrell
On Mar 16, 11:57 am, John Whittington <Joh...@mediscience.co.uk>
wrote:
>
> >Arbitrary thresholds result in gaming quite generally (take a look at
> >income tax brackets and clinical trial entry criteria). Not so clear
> >in diabetes but I'd wager that there are patients being treated who
> >have less disease than some not being treated. Sometimes this is just
> >because of how wealthy is the patient.
>
> Well, thankfully not in the UK, for reason's I've explained!
>
> However, the point surely is that 'patients being treated who have less
> disease than some not being treated' can only really happen with _your_
> approach. If the diagnosis were binary on the basis of fixed criteria, and
> if having the diagnosis automatically led to treatment (so, again, binary)
> (although that is not necessarily the case for Type II diabetes), then what
> you are saying could not happen. It undoubtedly does happen that some
diagnosis. If the diagnosis is not objective then it can be gamed.
> patients are treated who have less ('diabetes') disease than some who are
> not treated, but when this happens it is due to the very thing you seem to
> want to see - a multifactorial treatment decision in which factors other
> than the (binary) diagnosis of diabetes are influencing the decision
> whether or not to treat the diabetes. So, far from being undesirable
> 'gaming', what you describe sounds to me as if it's exactly what you like
> to see!
reduce this possibility.
>
> >This has led to major problems. Here is one example, though in a
> >clinical trial setting. A drug was ruled as ineffective just because
> >it reduced the number of serious MIs. Unfortunately for the sponsor
> >the main endpoint was all MIs (minor and major heart attacks combined)
> >and minor MIs washed out the treatment effect.
>
> Well, that's just a 'silliness' of unduly pedantic statistical/regulatory
> thinking. Hopefully the sponsor went on to undertake a subsequent study
> with a main endpoint suitable to demonstrate what the drug _could_ do.
>
> From what you say, I would suspect that what may well have been happening
> in that study was that the treatment was reducing the severity of serious
> MIs , turning them into less serious ones, (minor heart attacks' as you put
> it - although, as someone whose clinical years were mainly in/around
> cardiology, I think I would question the concept of a 'minor heart attack',
> not the least because the most 'trivial' {in terms of degree of coronary
> artery blockage or myocardial damage} have the potential to be fatal!),
> thereby leaving the total of minor+major relatively unchanged. Given that
> such an occurrence might have been predicted as a possible outcome, I think
> we're talking more of an imperfectly designed trial than anything directly
> to do with the way that diagnoses are classified. In reality, the sponsor
> may have considered that but, not knowing what the drug would do, thought
> that it was probably a 'better bet' to go for all MIs!
common problem, again caused by dichotomization.
>
> ...and, as I keep saying, I have absolutely no problem with subdivision (by
> severity, or anything else appropriate) of 'binary' diagnoses - which is
> what happens all ther time in clinical practice.
>
> >This is a mirage that does not help the patient in the long run.
> >Hundreds of thousands of patients have obtained "triple bypass
> >surgery" because of 3 non-significant coronary artery lesions that
> >caused their cardiologist to label the patient as having coronary
> >artery disease.
>
> That sounds like the US :-) Over here, resources for undertaking such
> surgery (at least, by the NHS) is so restricted that only the most
> deserving (high risk) cases are likley to get the surgery! Of course, we
> also have some wealthy people, and some of them undoubtedly cajole the
> private sector into performing surgery that would probably not be performed
> on an NHS patient!
>
> >Great dicussion; thanks for new topic heading. I hope this discussion
> >keeps returning to medstats. I think it is all-important, affecting
> >most of what we do in clinical biostatistics.
>
> Agreed - but, from some of the things you've written, I think that you
> probably underestimate the extent to which the approach you would like is
> actually applied in clinical practice (certainly in the UK), even if
> perhaps less formally than you would like.
Frank
Frank Harrell
its concoction Holmes was right. We don't know all the stories in
which he was wrong - that would not have held the reader's attention.
For every deduction (where the target is not in the room to
immediately confirm or refute) there is a probability. You think of
it as dichotomous only because you are censoring the failures.
Cheers
Frank
On Mar 16, 1:05 pm, "Thompson,Paul" <Paul.Thomp...@SanfordHealth.org>
wrote:
Frank Harrell
http://psychrights.org/articles/rosenham.htm
Frank
John Whittington
>I've never seen a trial where MIs were graded in the outcome. It's a
>common problem, again caused by dichotomization.
I'm somewhat amazed to see you suggest that.
As I briefly wrote in the other thread, one of the problems is in finding a
meaningful way of grading myocardial infarctions (MIs), since the seemingly
most trivial (minimal myocardial damage) can result in fatal arrhythmias -
so that a 'minor' MI may prove fatal, whereas a 'major' one may
not. However, that doesn't stop people trying, and there is a plethora of
literature relating to trials which have been undertaken with the outcome
being some measure of the 'severity' of the MI - usually based on the size
or haemodynamic/clinical consequences of the infarct, but sometimes other
measures such as biochemical. Indeed, the vast literature on trials
of 'acute intervention' (acute angioplasty or bypass surgery) generally
has some measure of the severity (e.g. size) of the ultimate infarct (if
any) following intervention as the main outcome, not to mention the
probably equally vast number of trials (over a much longer time frame)
which have studied the effects of various ways of management of MI on of
the ultimate infarct size/severity/consequences. That was already going on
(albeit with limited technological means of 'measuring' MIs) at least 40-50
years ago, in relation to the use of things such as hyperbaric oxygen,
anticoagulants, bed rest, sedation/anaesthesia etc. etc. in the management
of MIs; I actually witnessed one of the hyperbaric oxygen trials during my
training, around 1970.
Just try doing literature searches for things like 'myocardial infarction
size', 'myocardial infarction severity' etc.
Frank Harrell
Having worked in one of the best cardiology programs for 17 years I'm
well aware of those issues.
Imperfections in scoring systems does not mean that they are as bad as
dichotomization.
Frank
John Whittington
>Having worked in one of the best cardiology programs for 17 years I'm
>well aware of those issues.
As you will realise, I was simply responding to your statement....
>I've never seen a trial where MIs were graded in the outcome.
....which presumably cannot be what you really intended to write if you
have been involved with cardiology for all that time.
>Imperfections in scoring systems does not mean that they are as bad as
>dichotomization.
I never suggested otherwise. Indeed, all those trials studying
limitation/reduction of infarct size/severity/whatever would obviously have
been impossible if only a binary outcome (MI/No MI) were available.
I have to say that I'm getting a bit confused. First of all you suggest
that grading of MI outcome is never done, but now you are talking about
imperfections in the grading/scoring.
Frank Harrell
On Mar 17, 7:11 am, John Whittington <Joh...@mediscience.co.uk> wrote:
> At 19:58 16/03/2011 -0700, Frank Harrell wrote:
>
> >Having worked in one of the best cardiology programs for 17 years I'm
> >well aware of those issues.
>
> As you will realise, I was simply responding to your statement....
>
> >I've never seen a trial where MIs were graded in the outcome.
>
> ....which presumably cannot be what you really intended to write if you
> have been involved with cardiology for all that time.
scale to be used but the final choice in all randomized clinical
trials in which I was involved was to simplify the problem down to a
binary outcome.
Frank
John Whittington
>On Mar 17, 7:11 am, John Whittington wrote:
> > As you will realise, I was simply responding to your statement....
> > >I've never seen a trial where MIs were graded in the outcome.
> > ....which presumably cannot be what you really intended to write if you
> > have been involved with cardiology for all that time.
>
>That's not the case John. I lobbied many times for an ordinal outcome
>scale to be used but the final choice in all randomized clinical
>trials in which I was involved was to simplify the problem down to a
>binary outcome.
Fair enough. You are obviously talking about your own experience of
specific trials. However, as I've explained, in the wider world, trials in
which the MI outcome is non-binary (often 'continuous', not merely ordinal)
are very common.
In fact, returning to what appears to be your underlying philosophy, I
would question the appropriateness of any ordinal classification of MIs -
which would be both difficult and essentially arbitrary. Given many
possible continuous measures of MI extent and consequences, it would seem
far better to use all the available data, rather than wasting some
information by collapsing it to an ordinal scale - particularly one as
crude and arbitrary as 'minor'/'major'.
Another factor, which may account for your lobbying having been
unsuccessful in relation to the trials you have been involved with, is that
the _interest_ has actually been in the binary question, making that the
appropriate outcome measure. I think what one needs to realise is that,
once one has had an MI, the 'severity' (whether in terms of extent of the
infarction or its consequences) of that MI is very much a matter of
chance. Particularly if someone has generalised or widespread coronary
artery disease, so much depends upon where a total blockage eventually
happens to arise (particularly in relation to the branching) - and that is
very largely a chance thing. Similarly, whether or not a given MI (whether
small or large) leads to electrical disturbances and consequent potentially
fatal arrhythmias is again very largely a matter of chance. This all leads
to doubt as to whether there would, in many studies, be any real clinical
usefulness in having MIs classified according to some arbitrary
classification (e.g. 'minor'/'major', or a wider classification), rather
than just present/absent (according to defined criteria).
Thompson,Paul
In such models, the 0-1 (no event - event) aspect is considered first, and after that, we consider magnitude of the event in the "event" cases.
If you were considering selling tickets to a concert, some will be interested in the concert and some will not. For those who are not interested, it is very silly to ask their opinion of the amount of money for the tickets - they are totally uninterested. We could consider the amount those willing to go would spend. Only a person willing to go to the event is a reliable informant about ticket prices.
Similarly, with MI, factors such as BMI, lipid levels, and BP are considered to be implicated in the processes which lead to the occurrence of the MI. They may or may not operate in the same ways for the different levels of the MI.
Infections also fall into this.
Much of the discussion seems to be confusing the "process which leads to event" with "process which creates an event which is larger or smaller".
Paul A. Thompson, Ph.D.
-----Original Message-----
From: meds...@googlegroups.com [mailto:meds...@googlegroups.com] On Behalf Of John Whittington
Sent: Thursday, March 17, 2011 7:12 AM
To: meds...@googlegroups.com
John Whittington
>In this area, and in many others, the notion of the "two-part model" seems
>appropriate.
>In such models, the 0-1 (no event - event) aspect is considered first, and
>after that, we consider magnitude of the event in the "event" cases.
Maybe - although, as I've been saying, it may well be that for many trials
there is actually only interest in the first part, not the second.
>Similarly, with MI, factors such as BMI, lipid levels, and BP are
>considered to be implicated in the processes which lead to the occurrence
>of the MI. They may or may not operate in the same ways for the different
>levels of the MI.
Whilst I agree with the general concept, as I've been trying to say, I
think that MI is probably a bad example to be using, since the 'level'
(size, seriousness, consequences, fatality etc.) is very largely determined
by essentially chance factors.
On a personal level, I can tell you that I would be as scared as hell if I
were told that I had just had a 'minor MI' (on the basis of whatever
classification) - since, over the years, I have seen far too many people
die minutes, hours or days after suffering an MI which some people would
classify as 'minor' (until it resulted in death!).
With many other diseases and disease process, the concept of grades of
severity is far more meaningful, and useful.
Frank Harrell
On Mar 17, 7:58 am, John Whittington <Joh...@mediscience.co.uk> wrote:
> At 05:29 17/03/2011 -0700, Frank Harrell wrote:
>
> >On Mar 17, 7:11 am, John Whittington wrote:
> > > As you will realise, I was simply responding to your statement....
> > > >I've never seen a trial where MIs were graded in the outcome.
> > > ....which presumably cannot be what you really intended to write if you
> > > have been involved with cardiology for all that time.
>
> >That's not the case John. I lobbied many times for an ordinal outcome
> >scale to be used but the final choice in all randomized clinical
> >trials in which I was involved was to simplify the problem down to a
> >binary outcome.
>
> Fair enough. You are obviously talking about your own experience of
> specific trials. However, as I've explained, in the wider world, trials in
> which the MI outcome is non-binary (often 'continuous', not merely ordinal)
> are very common.
prevention trials.
>
> In fact, returning to what appears to be your underlying philosophy, I
> would question the appropriateness of any ordinal classification of MIs -
> which would be both difficult and essentially arbitrary. Given many
> possible continuous measures of MI extent and consequences, it would seem
> far better to use all the available data, rather than wasting some
> information by collapsing it to an ordinal scale - particularly one as
> crude and arbitrary as 'minor'/'major'.
use it. Cardiologists show strong agreement in construction of
ordinal scales but seldom end up using them. A 5-level ordinal scale
with each level well populated can give most of the information in
some cases.
>
> Another factor, which may account for your lobbying having been
> unsuccessful in relation to the trials you have been involved with, is that
> the _interest_ has actually been in the binary question, making that the
> appropriate outcome measure. I think what one needs to realise is that,
> once one has had an MI, the 'severity' (whether in terms of extent of the
> infarction or its consequences) of that MI is very much a matter of
> chance. Particularly if someone has generalised or widespread coronary
> artery disease, so much depends upon where a total blockage eventually
> happens to arise (particularly in relation to the branching) - and that is
> very largely a chance thing. Similarly, whether or not a given MI (whether
> small or large) leads to electrical disturbances and consequent potentially
> fatal arrhythmias is again very largely a matter of chance. This all leads
> to doubt as to whether there would, in many studies, be any real clinical
> usefulness in having MIs classified according to some arbitrary
> classification (e.g. 'minor'/'major', or a wider classification), rather
> than just present/absent (according to defined criteria).
nebulous (sometimes it's just P < 0.05 at whatever you can lay your
hands own, but not when talking to the most knowledgeable
cardiologists) and is considered with reference to tradition and
acceptability and interpretation by non-experts. But just speaking in
general terms, almost anything is better than a binary classification,
in terms of power, precision, and public health utility.
I've already said more than I know about these issues so am signing
off for now. I might come back later with a proposal for diagnostic
research that tries to summarize some of the general principles.
Regards,
Frank
Frank Harrell
I have proposed this in an NIH-sponsored clinical trial that was not
funded. Here's a key reference:
@ARTICLE{ber91ana,
author = {Berridge, D. M. and Whitehead, J.},
year = 1991,
title = {Analysis of failure time data with ordinal categories of
response},
pages = {1703-1710},
annote = {time and severity of event}
}
Cheers
Frank
On Mar 17, 9:03 am, "Thompson,Paul" <Paul.Thomp...@SanfordHealth.org>
wrote: