Why Number Needed to Treat Can Be Misleading for Vaccines

23 views
Skip to first unread message

Marc Schwartz

unread,
May 5, 2021, 7:42:52 AM5/5/21
to MedStats
Hi All,

Thought that you would find this commentary by Andrew Althouse of interest:

  Why Number Needed to Treat Can Be Misleading for Vaccines
  https://www.medscape.com/viewarticle/950117

It appears to be open access if you do not have a Medscape account.

Regards,

Marc Schwartz

John Whittington

unread,
May 5, 2021, 9:30:24 AM5/5/21
to meds...@googlegroups.com
Thanks Marc.  It seems to me that what he says is totally correct.

I haven't yet (but will) follow his links to find out what the 'contrary viewpoints' are, but is seems pretty obvious to me (for the reasons he gives, which are easy enough to understand) That ARR and  NNT are not appropriate concepts in the context of the trial designs we are talking about - i.e. involving short-term exposure to relatively low levels of risk, and with event-driven analyses.

Kind Regards,
John

John

----------------------------------------------------------------
Dr John Whittington,       Voice:    +44 (0) 1296 730225
Mediscience Services       Fax:      +44 (0) 1296 738893
Twyford Manor, Twyford,    E-mail:   Joh...@mediscience.co.uk
Buckingham  MK18 4EL, UK            
----------------------------------------------------------------

John Whittington

unread,
May 5, 2021, 6:00:37 PM5/5/21
to meds...@googlegroups.com
Further to what I wrote earlier (below), it has occurred to me that there is perhaps another way one could look at, and express, this ...

... I don't think there is anything fundamentally 'wrong' with the concept of NNT in relation to the vaccine trials, provided that one defines what it means in the context of those trials - but it is if/when one produces that definition that one realises how relatively 'unhelpful' such an 'NNT' is ...

As I see it, in the context of the vaccine trials, what the calculated 'NNT' essentially means is "the number of people who would have to be vaccinated in order to prevent one infection occurring during a relatively short period of time during which exposure to virus was relatively low".

... whereas, what we are far more interested in knowing is "how many would have to be vaccinated in order to prevent one infection occurring (ever) in a population with continuing high exposure to the virus".

Hence, perhaps not so much 'incorrect' as 'the correct answer to the wrong question'?

That's how I see it, anyway.

Kind Regards, 
John
--
--
To post a new thread to MedStats, send email to MedS...@googlegroups.com .
MedStats' home page is http://groups.google.com/group/MedStats .
Rules: http://groups.google.com/group/MedStats/web/medstats-rules

---
You received this message because you are subscribed to the Google Groups "MedStats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to medstats+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/medstats/202105051330.145DUILC005811%40mail102c50.megamailservers.eu .

SteveDrD

unread,
May 6, 2021, 7:31:49 AM5/6/21
to MedStats
Classic Type III error 

John Whittington

unread,
May 6, 2021, 7:54:24 AM5/6/21
to meds...@googlegroups.com
At 12:31 06/05/2021, SteveDrD wrote:
Classic Type III error

Quite so.  For any who are not familiar with that terminology (I wasn't until relatively recently!), if I understand correctly you are essentially referring to my closing comment below which read ...


Hence, perhaps not so much 'incorrect' as 'the correct answer to the wrong question'?

Kind Regards, 
John

On Wednesday, May 5, 2021 at 6:00:37 PM UTC-4 John Whittington wrote:
Further to what I wrote earlier (below), it has occurred to me that there is perhaps another way one could look at, and express, this ...

... I don't think there is anything fundamentally 'wrong' with the concept of NNT in relation to the vaccine trials, provided that one defines what it means in the context of those trials - but it is if/when one produces that definition that one realises how relatively 'unhelpful' such an 'NNT' is ...

As I see it, in the context of the vaccine trials, what the calculated 'NNT' essentially means is "the number of people who would have to be vaccinated in order to prevent one infection occurring during a relatively short period of time during which exposure to virus was relatively low".

... whereas, what we are far more interested in knowing is "how many would have to be vaccinated in order to prevent one infection occurring (ever) in a population with continuing high exposure to the virus".

Hence, perhaps not so much 'incorrect' as 'the correct answer to the wrong question'?

That's how I see it, anyway.

Kind Regards, 
John


At 14:32 05/05/2021, John Whittington wrote:
Thanks Marc.  It seems to me that what he says is totally correct.

I haven't yet (but will) follow his links to find out what the 'contrary viewpoints' are, but is seems pretty obvious to me (for the reasons he gives, which are easy enough to understand) That ARR and  NNT are not appropriate concepts in the context of the trial designs we are talking about - i.e. involving short-term exposure to relatively low levels of risk, and with event-driven analyses.

Kind Regards,
John

At 12:42 05/05/2021, 'Marc Schwartz' via MedStats wrote:
Hi All,

Thought that you would find this commentary by Andrew Althouse of interest:

Why Number Needed to Treat Can Be Misleading for Vaccines


It appears to be open access if you do not have a Medscape account.

Regards,

Marc Schwartz

John

----------------------------------------------------------------
Dr John Whittington,       Voice:    +44 (0) 1296 730225
Mediscience Services       Fax:      +44 (0) 1296 738893
Twyford Manor, Twyford,    E-mail:   Joh...@mediscience.co.uk
Buckingham  MK18 4EL, UK            
----------------------------------------------------------------

--
--
To post a new thread to MedStats, send email to MedS...@googlegroups.com .
MedStats' home page is http://groups.google.com/group/MedStats .
Rules: http://groups.google.com/group/MedStats/web/medstats-rules

---
You received this message because you are subscribed to the Google Groups "MedStats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to medstats+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/medstats/202105051330.145DUILC005811%40mail102c50.megamailservers.eu .

John

----------------------------------------------------------------
Dr John Whittington,       Voice:    +44 (0) 1296 730225
Mediscience Services       Fax:      +44 (0) 1296 738893
Twyford Manor, Twyford,    E-mail:   Joh...@mediscience.co.uk
Buckingham  MK18 4EL, UK            
----------------------------------------------------------------

--
--
To post a new thread to MedStats, send email to MedS...@googlegroups.com .
MedStats' home page is http://groups.google.com/group/MedStats .
Rules: http://groups.google.com/group/MedStats/web/medstats-rules

---
You received this message because you are subscribed to the Google Groups "MedStats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to medstats+u...@googlegroups.com.

John Whittington

unread,
May 6, 2021, 9:59:24 AM5/6/21
to meds...@googlegroups.com
Apologies for adding another post, but I have been thinking yet more and it has occurred to me that there is probably a more general problem with using the normal concept of NNT ("Number Needed to Treat") in the context of a vaccine, EVEN IF one addresses the issues I mentioned below by conducting a trial with a long period of observation (ideally the full duration of the vaccine's effectiveness) and in people exposed to a relatively high risk of infection.

In other situations, NNT is straightforward (and useful) enough, because it represents the number of people/patients who have to be treated in order to achieve one 'successful result' (e..g. prevention of some outcome/disease or successful treatment of a disease), in one person/patient.

However, in the case of the vaccine, if (in the context of the prevailing degree of population susceptibility/immunity) the "R" figure is greater than unity, then "one case of disease prevented" (in trial participants) can translate to many cases of disease prevented (primarily in people NOT participating in the trial, hence 'not counted').  For example, data from trial participants may indicate that one needs to treat, say, 100 people in order to prevent one case of infection in trial participants (hence NNT=100), but if that one prevented infection resulted in, say, 49 more 'prevented cases' in the wider (non-trial) population, then the 'true NNT' would be only NNT=2.

Does that make sense?

Kind Regards,
John

Martin Holt

unread,
May 21, 2021, 8:21:35 AM5/21/21
to MedStats
GPower, MedCalc, "Sample Size Tables for Clinical Studies" 2nd Edn by David Machin, Michael Campbell, Alain Pinol!
Reply all
Reply to author
Forward
0 new messages