Hello decision group,
Some of you may perhaps be interested in either or both of my two new articles published this month in
Synthese and Theory and Decision. Both relate econometric analysis of partial identification with decision theory, for different audiences. Here are links and abstracts:
Controversy about the significance of underdetermination of theories persists in the philosophy and conduct of science. The issue has practical import when research is used to inform decision making, because scientific uncertainty yields inductive risk. Seeking
to enhance communication between philosophers and researchers who study public policy, this paper describes econometric analysis of partial identification and its use in welfare-economic policy analysis. Study of partial identification finds underdetermination
and inductive risk to be highly consequential for credible prediction of important societal outcomes and, hence, for credible public decision making. It provides mathematical tools to characterize a broad class of scientific uncertainties that arise when available
data and well-supported assumptions are combined to predict population outcomes. Combining study of partial identification with criteria for reasonable decision making under uncertainty yields coherent approaches to make policy choices without accepting one
among multiple empirically underdetermined theories. The paper argues that study of partial identification warrants attention in philosophical discourse on underdetermination and inductive risk.
Enormous attention and resources are being devoted to the quest for artificial general intelligence and, even more ambitiously, artificial superintelligence. We wonder about the implications for methodological research that aims to help decision makers cope
with what econometricians call identification problems, inferential problems in empirical research that do not diminish as sample size grows. Of particular concern are missing data problems in prediction and treatment choice. Essentially all data collection
intended to inform decision making is subject to missing data, which gives rise to identification problems. Thus far, we see no indication that the current dominant architecture of machine learning-based artificial intelligence (AI) systems will outperform
humans in this context. In this paper, we explain why we have reached this conclusion and why we see the missing data problem as a cautionary case study in the quest for superintelligence more generally. We first discuss the concept of intelligence, focusing
initially on some work by AI researchers, before presenting a decision-theoretic perspective that formalizes the connection between intelligence and identification problems via the general ability to specify a realistic state space. We next apply this perspective
to two leading cases of missing data problems. Then we explain why we are skeptical that AI research is currently on a path toward machines doing better than humans at solving these and other identification problems.
Chuck Manski