Thanks for the comments Alex. You mentioned statistical interpretation
which actually is quite confusing as to how it relates to this
article. I was worried for a little bit about whether my point has
been nullified, but after some more thought, I think I'm safe.
So let us say there are two competing theories for explaining this
picture. Theory 1 says that it is a dalmatian. Theory 2 says that it
is a clown.
I personally think Theory 1 makes more sense, and as you said, would
think so even if you insisted otherwise. However, I cannot think of
any objective reason why Theory 1 is a "better" theory than Theory 2.
It just makes more sense to *me*. How exactly would you demonstrate
that a dalmatian explains more of the picture than a clown does? By
what and whose objective function?
In response to your point about statistical interpretation, I believe
that they run into the exact same problem actually. Which is that,
having already observed the data, there isn't a satisfying way to say
that one model is better than another. This leads to the classical
problem of "overfitting". There are heuristics that people end up
using (eg. regularizers, or cross-validation), but they *are*
heuristics and which one you use is a matter of personal choice.
(Otherwise we would all end up using the same thing and model
selection would be a solved problem.)
I would like to say, that the problem of being given a 2d image of
black and white pixels, and being asked "What is it a picture of?" -
is analogous to the problem of being given a set of points and being
asked "What order polynomial are these points drawn from?". And we
just don't have a good answer to that question.
An example of a similar but objective problem would be the computer-
vision problem you mentioned where we have a big collection of
pictures, got a hundred people to figure out what is in them, and our
*objective* is to come up with a classifier that agrees with as many
people as possible. That is a perfectly objective task, where noisy
inputs is an added difficulty. But that is not the reason I prefer
Theory 1. I prefer Theory 1 simply because it makes sense to me. Even
if a thousand other people and computer-vision algorithms insist
otherwise, I would still prefer Theory 1. Wouldn't you?
On Sep 23, 11:52 am, Alexander Farley <
alexander.s.far...@gmail.com>
wrote: