SummerSim 2017: Using Sensitivity analysis to examine the effects of an Ebola vaccine - Reviewer Mike Famulare
Jacob Barhak
Here is my review of “Using Sensitivity analysis to examine the effects of an Ebola vaccine” by Wiafe, Veilleux-Gravel, and Smith?.
Summary and recommendation: Borderline
(Note that while I am an experienced epidemiological modeler, this is paper is my first exposure to SCS, and so my comments on points of style and scholarship may not reflect the norms of the SCS community. Nonetheless, I look forward to attending my first SCS conference and appreciate the opportunity to review for your organization.)
This paper highlights the dominant roles contact behavior and (a lack of) immunity play in ebolavirus outbreaks and studies them in the context of a bifurcation analysis around the endemic equilibrium of a deterministic ordinary differential equation (ODE) model. The results emphasize the importance of understanding the durability of vaccine-derived immunity on assessing the impact of an ebolavirus vaccine on endemic transmission. The introduction serves as a useful quick primer to non-specialist audiences on the outcome and impact of the recent ebolavirus outbreak. The methods are quickly but clearly explained. The mathematical results appear to be correctly derived from within the ODE model context.
However, there are problems with the framing and analysis in the context of the recent ebolavirus outbreak. First, the authors discuss ebolavirus eradication. This is inappropriate as “eradication” has a specific meaning in public health policy (Dowdle 1998) and eradication is not possible for zoonoses unless the pathogen is also eliminated from animal reservoirs. A more appropriate way to frame this question is to ask “under what conditions is the elimination of an ebolavirus strain circulating within a human population likely?”.
More importantly, analysis of eradication from an endemic equilibrium with a deterministic model is not appropriate for a disease that is dominated by stochastic zoonotic outbreaks with dynamics far from equilibrium. Their treatment ignores the transient immune dynamics during outbreaks that make post-outbreak elimination more likely in finite populations. The authors in the abstract and discussion note the relevance of stochastic effects, but do not quantify them. (A recent paper that the authors couldn’t have known about prior to submission examines this for ebolavirus.) Also, by assuming endemic equilibrium with Ro near 1, the authors implicitly assumed that the mean age of infection in an unvaccinated population is comparable to a typical human lifetime (exponentially-distributed in the model) (Keeling and Rohani p 33), and thus they also implicitly assumed their conclusion that immunity would need to persist for 50 years to guarantee elimination of an endemic strain.
If this was a journal publication, I would recommend rejection as the analysis as presented isn’t appropriate to the disease as currently framed by the authors. However, as this is a conference proceeding, a lower bar for work in progress is reasonable, and so my recommendation is Borderline: acceptance at the discretion of the conference organizers. Language clarifying the meaning of “eradication” in the context of human ebolavirus outbreaks can be fixed, but a stochastic model for realistic finite population sizes (or at least an ODE in the post-outbreak transient regime) is more appropriate than an endemic ODE model for studying ebolavirus elimination. Minimally, I would recommend reframing the analysis as applying in contexts where an ebola-like pathogen escapes early control efforts prior to vaccine development.
Major comments
How was waning effect = 0.02 chosen to define “can be theoretically eradicated?” There are samples of Ro>1 at lower values of the waning effect in Figure 4.
I don’t understand how there exists a disease-free equilibrium in this model with finite c_m (exposure to infected meat). Because c_m is very small (new zoonotic outbreaks are rare), it is reasonable to discuss the equilibrium in the c_m goes to zero limit (especially as the propagation of this exposure to a measurable outbreak is inherently stochastic such that the ODE equilibrium isn’t meaningful anyway). If this is the assumption being made, the paper needs to state it, and explain what (if any) role is played by c_m in the analysis because it is described as a relevant parameter in the methods.
From the abstract and methods, I expected that this paper was addressing an interesting question that is often overlooked in setting vaccination policies: under what conditions can introducing a vaccine increase long-term disease burden over the unvaccinated state? However, this perverse behavior is only briefly mentioned in the discussion as not relevant for ebolavirus. If this point is important enough to remain in the paper, I would like to see the results for analyzing vaccinated contact behavior. How large an increase in contact rates would be required at what levels of vaccination coverage for perverse outcomes to occur?
The figures need explanatory captions.
Minor comments
Top of page 5 and throughout: Replace “the vaccine may wane over time” with “immunity from vaccination may wane over time” and similar usages.
Page 10: Numerical simulations: is it 1000 times per parameter or 1000 times for all parameters?
Figure 4-6: bigger dots. The current symbols are distractingly small when viewed on my monitor.
In general, I think the information in figures 3-6 would be better represented as two bivariate scatter plots with Ro as the color (see Fig 1 of McCarthy et al for example). This would show the correlations of waning, contact, and transmission as well as their effects on Ro in a manner more informative to the reader.
Reference 18 is missing “HIV” from the title.
__________
I’m willing to engage further with you and the authors for any desired follow up. I declare no conflicts of interest.
Thank you,
--Mike
Mike Famulare, Ph.D.
Senior Research Scientist
Institute for Disease Modeling
Jacob Barhak
Hi Jacob,
I think the revised paper is acceptable for publication pending one necessary
revision to the figures. I also have some additional comments that may improve
it further, but I don’t consider them blocking. I appreciate the authors’
efforts to improve the manuscript.
Necessary revision:
The axes labels on the figures 2-5 (and transition parameters in Figure 1) must
be larger so they can be read. Only Figure 3A is fine as-is.
Other suggestions:
Some paragraphs in the intro are not well-linked to the model and results, and
so it is difficult to see where the paper is going from its intro. While
interesting, I suggest removing the first 4 paragraphs of page 2 about
non-specific effects on mortality, poverty, and children, and streamlining the
rest of the intro to stay close to modeling goals. If the intro is intended to
provide general background to an audience unfamiliar with infectious disease,
it would help if the general background parts were tightened up, and if the
more relevant parts of the intro were called out as directly relevant.
End of section 2, “perverse outcomes” of vaccination: It would be useful to
have a sentence near the definition of perversity that gives an example of when
R_v might be higher than R_0 (such as behavior change as is described in the
discussion).
Thank you for the interesting read,
--Mike Famulare
Mike Famulare, Ph.D.
Senior Research Scientist
Institute for Disease Modeling
Jacob Barhak
I agree with Zvi. Also, I like this review process. Thank you for organizing,
--Mike
Mike Famulare, Ph.D.
Senior Research Scientist
Institute for Disease Modeling