Last year the entire panel largely reduced to everyone arguing that hard problems
require trading off bias for reduced variance, then my countering that you can’t do
that robustly in practice without being able to quantify the bias and variance.
Some agreed in concept but perhaps not in practice, some argued that no one has
ever done that for any algorithm so we why start now? The conversation hasn’t
evolved much since then so I don’t really have anything new to contribute.
But if the panel this year is focused on software then maybe someone can talk about
setting up model ensembles for benchmarking algorithm accuracy and some of
the early results with Stan and ADVI. Of course if the focus is on tools for the
applied researcher it would be great for the speaker to emphasize that applied
researchers can do only fragile work when they get nominal results without any
sense of error!