Upcoming session on sensitivity analysis: Association for Public Policy Analysis and Management (Seattle, Nov 14)
Ken Frank
Association for Public Policy Analysis and Management (APPAM)
https://convention2.allacademic.com/one/appam/appam25/
November 14, 2025 (1:45–3:15pm Pacific time). In person: Seattle, WA.
Description:
The session informs discussion about the strength of evidence supporting a policy or practice by quantifying the conditions necessary to change an inference. In part I, we use the potential outcomes and counterfactual framework to interpret how much bias there must be to nullify an inference. This supports statements such as “To nullify the inference, X% of the data would have to be replaced with counterfactual cases for which there was no effect.” In part II, we quantify the robustness of causal inferences in terms of correlations associated with unobserved variables. This supports statements such as “To nullify the inference an omitted variable would have to be correlated at __ with both the focal predictor and outcome.” By leveraging intuition based on potential outcomes and correlations with omitted variables, these techniques inform collaborative evidence generation and evaluation among a broad set of stakeholders. Furthermore, because the techniques are functions of quantities commonly reported in published studies (e.g., estimated effect, standard error) they can be broadly applied and extended (e.g., logistic regression, hazard, propensity score based). Specifically, the techniques have already been widely employed in policy, education, sociology, epidemiology, and management, as well as in general science (e.g., Proceedings of the National Academy of Sciences).
We introduce participants to calculations using modules in Stata and R, as well as the app (https://konfound-project.shinyapps.io/konfound-it/). There will be a mixture of presentation and individual hands-on exploration. Participants will also have an opportunity to compare with alternative techniques (e.g., Oster’s Coefficient of Proportionality; Cinelli & Hazlett’s Robustness Index). This workshop is intended for anyone familiar with the general linear model but also informs more sophisticated techniques (e.g., propensity scores, logistic regression, multilevel models).